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Abstract 



The Vim computer system, an experimental project under development in the Computation 
Structures Group at MIT, is intended to examine the efficient implementation of functional 
languages using the principles of data flow computation. In this thesis, we examine how to 
incorporate backup and recovery mechanisms into this system to guarantee that no online 
information is lost because of hardware malfunction. Our solution, which takes advantage of Vim's 
powerful applicative base language and its uniform treatment of data and files, integrates the 
operation of the backup and recovery system within the interpreter itself, resulting in a system that 
can ensure a high degree of data security without excessive performance degradation. Unlike 
schemes found in other systems to guarantee data security, operation of the backup facility requires 
no user intervention. 

To present our algorithms rigorously, we first develop a formal operational model of system 
behaviour. This set-theoretic model views Vim as a state transition system with the interpreter 
serving as a state transition function. The specification language is a superset of the applicative 
language, VimVal. We enhance this model to include a concept of system failure and augment to 
the basic components in the system a backup state with the base language instructions now 
operating on both the Vim as well as the backup state. A formal proof demonstrating the 
correctness of our algorithms is also given. Issues concerning the implementation of these 
algorithms are also addressed in the diesis. 
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Title: Professor of Electrical Engineering and Computer Science 

Keywords: Dataflow, Dynamic Dataflow Architecture, Functional Languages, Fiult 
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2 GOALS OF THE THESIS §1.1 

found in our system provides a framework on which a highly secure system can be designed. 
Unlike its more conventional counterparts, the backup and recovery utilities we present are 
simple and efficient and do not require any r i|fef gguridance to perform their task. We model the 
presence of failures and the actions undertaken by the backup and recovery mechanisms by 
defining a formal operational semantics of system behaviour, this formal model is used to give a 
precise description of the behaviour of the backup and recovery system. We demonstrate, using 
this formal model, that the backup algorithms developed properly records the relevant portions 
of the system state and mat the recovery system does correctly restore the proper system state as 
well. 

1.2 Motivation 

The problem of guaranteeing data security is certainly not a new one. Virtually every 
major computer system developed includes some type of protection mechanism to safeguard the 
information entrusted to U. It is. therefore, natural for Ihe reader tejquestion why the study of 
data security for the Vim system is a problem worthy of investigauon. There are two main 
reasons why we have not chosen to simply use backup and recovery algorithms developed for 
other systems for Vim. First, and foremost, backup and recovery systems in conventional 
systems are not able to provide full data security without beeommg excessively complex and 
inefficient In general the implementations of these uiihties are not able to prawd© futt data 
security without incurring significant reduction in system performance. In addition to this major 
drawback, these schemes are based on a computational model which is much different from that 
found in Vim. The implementation strategy that we present in this thesfe guarantees full data 
security and exploits the novel features of Vim in doing so. it isagniftcantly different from other 
data security schemes proposed for both sequential and pamMel systems. Some of the more 
interesting aspects of Vim pertinent to the issue of data security are cited below: 

•Vim is based on a dynamic data flow architecture. 

In a data flow computer system, all instructions in, the program are viewed as potentially 
executable, with the only constraint being that they must have received all necessary operands 
and control signals. This execution model allows fpr a great .deal of concurrency to be rcaljzed. 
Thus, our data security mechanism must be designed to be used in a highly concurrent 
environment. 
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•Vim interprets an applicative base language, namely, the language of dynamic data 
flow graphs. 

A dynamic dataflow graph is a directed acycKc graph where nodes represent instructions and arcs 
define data dependencies between these instructions. A^grajih is dynamic if the execution of a 
function activation is explicitly initiated by torn* epplfopetme. In bur system, each ftmction 
activation has its own graph created by tiketppCp instruction; Th^base^anguage instruction set is 
very powerful, containing instructions for function application and retiul^stiuctiire creation and 
manipulation as well as instructions to handle storage management. We can take advantage of 
this feature in the design of our data security mechanism by enhancing die semantics of these 
instructions to support the backup and recovery procedures. 

•There is no distinction between./?/?* and data in Vim, 

Unlike conventional systems, the units of storage allocation in VrM are the information units on 
which the primitive operations of Vim operate £ a irtstntetions, scalar Valiiei arrays, and records. 
This feature will allow us to design backup aigorfih1&#Mt^'be^ of how 

information is being created and manipulated mthiBfsysOtot 

•The unit of transfer between disk and main memory is a small fixed-size unit of 
information called a chunk. 

The small unit of information transfer will allow extensive concurrency of operation to be 
achieved by exploiting the data driven execution mdSeltestipporl 'it high levd of information 
traffic between disk and main memory. The use^Of cftunli* as the ttn» of transfer poses 
interesting problems for the backup system when mfeftnatierf needs iffbe copied frorri memory 
to a more stable storage medium. We discuss this itetie-fatefcin me thesis. 

1.3 Background 

While hardware is generally reliable for the most part, catastrophes do indeed occur and it 
is necessary that the computer system designer be sensitive to th«reality t? 4dcaUy, we would like 
to provide a guarantee to the users of our system that all acc^bje data, wjlj survive the effects of 
any hardware malfunction. The approach that is adopted, hpweyc^wijJi naturally be strongly 
influenced by efficiency constraints. In most con ventiona) computer systems that do not provide 
extra hardware support to achieve this aim, the cost of realizing full data security usually involves 
too much overhead and reduced performance to be a realistic goal. In Uicsc systems, users are 
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4 BACKGROUND §1.3 

forewarned that some of the information they have emrusted to &e system may not survive a 
hardware failure. Consequently, these users must take explicit action to preserve data they deem 
important by periodically "backing up" their inhumation onto arrtoee liable .storage medium. 
Information which is not transferred onto backup store is tasfcatto a crash. Jt is usually not 
possible in most systems to recover Oris tost data by rcexecuting the compirtations which initially 
produced it since there is no mechaaisa>to monitor the creation andaitodfrffiatioa of mfortnatton 
occurring in die system. 

We shall say that the information held by a computer system is fully secure from loss or 
corruption if the system contains mechanisms which ensure the preservation of all data despite 
the presence of unreliable underlying hardware components, these mechanisms may be 
incorporated as backup and recoveittoftware utffiti&df rriay ikl^he fbrrri bf replicated storage 
elements or may be implemented, as spine gwtfp^tf.pQtifr^jfe degree of date security 
provided by a computer system^ one n^ureo/i^*^ f 

the system can confidently s^re and, a^ses? data^ wte/iever desired. . Note that; data security does 
not necessarily imply reliability ar^.aseoir* 

external world. A system which does provide full data seetirftyjj however, guarantees its users 
that no failure in the system will cause any accessible infbrmWkft, to be tost even though users 
may be prevented from accessing it for a time if a failure takes place. 

The extent to which users can accew data when they w^is a measure of the availability of 
the system,, w, application^ such as rwiet guidance^ fee ejrtwle. it is cruciai that no single 
hardware failure makes the system u^aiplabl^ .6»«;i^ i M*Mr9«dMhofrjti^.--rl3MAMic 
approach to masking failwes of ku-dwa*e from th*e*te«aJ sy«e»^ best done at the hard ware 
level itself, by incorporating sufficient redundancy into the system {26]. In this thesis, we shall 
not be concerned with availability. "Rather, we shall be focusing our attention on the problem of 
enhancing VtM to provide full data security for alton^jpifbjinafcotk 

For systems Which need not mask failures, the standard approach used to guarantee data 
security has been to provide backup and nwi^ so^areutftt^ The backup 

utility serves to safeguard 1hf6^ c ff ccts f 

hardware failure. The recovery utility is invoked wtien a hardware failure occurs and uses the 
information preserved 'by the backup facility 'to r^reme system to a suite which existed before 
the failure. 'the degree of data security provided by 'tKe system 's dicLitccl by the cost of the 
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backup and recovery process. Most current systems have trad to sacrifice the goal Of providing 
full data security because of the high overhead that would be involved in supporting the backup 
and recovery system. In these systems, the state restored by thrreeovery utility may not be the 
one which existed immediately prior to the failure; Moreover itis usually the case that there is 
no means of recovering this desired state from the one produced by the recovery procedure. 
Thus, information loss is a possibility which users ofthe system must accept 

1.4 Previous Works 

In this section, we briefly describe some previous efforts in the design of highly secure 
systems undertaken for both centralized and distributed^systert^a^ weU as solutions proposed for 
providing fault-tolerance for a class of data flow architecauiip, ^ point out som^ major 
deficiencies and assumptions made in these proposals that make then} not suitable for 
implementation in our system. 

1;4.1 Data Security in Centralized Systems 

In conventional singie-processor machines," the simplest and perhaps most direct means of 
recording state reformation for recovery purposes is to eslaWiffo a checkpoint des^rjbirig all 
aspects of the system state. The checkpoint state is construcledby recording the stole of the 
system at some point onto a reliable storage medium such as, tape. The most obvious drawback 
of this scheme is the potentially large amount of data which must be examined - an oftentimes 
unnecessary and expensive strategy since most ofthe objects in me system would probably not 
have been modified between successive checkpoints. To ameliorate this problem somewhat, 
many timesharing systems perform an activity referred to as incremental dumping to ke^p the 
backup system abreast of any modifications to the file hierarchy between successive checkpoints. 
In the event of a major mishap that necessitates the reconstruction ^fu>© fil% hiecswhy^ljie 
system can be reloaded from the last checkpoint and appropriately modified by using the current 
incremental dumps to restore the system to a more recent sute. rt All inc,rejnental dumps are 
copied onto magnetic tape. In order that the recovery phase of state restoration may proceed 
faster, incremental dumps are periodically consolidated to remove oujdated copies of files. This 
consolidation process is known as secondary dumping. 

Using incremental and secondary dumps to record infermatiori tibout the system state is 
me basic approach used by the Multics operating system {27} to provide secure service to its 
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6 PREVIOUS WORKS §1.4 

users. A modification to the backup and recovery protocol employed by Multics was suggested 
by Benjamin [6] for the Data Network Computer, instead of using magnetic tape as the medium 
to store backup copies, it was advocated that the cotejwtefsysieiilje integrated withte a network 
of autonomous systems with each system being altewed access as lhe«torage<tevfces of the other 
processing elements. The backup facility maintains a consistent image of die file storage at a 
remote site within the network. The motivation for having such a backup system is the greater 
ease in managing the backup system that results when we do not have to deal with sequential 
access tape storage. The problem in using such a system, however, come^from ^ the decrease in 
availability that may occur due to the extra dependency on eornmtosication lines etc. 

Perhaps the most serious objection to the solution adopted by Multics is the cost involved 
in periodically scanning the file hierarchy to find those files which have been created or updated 
since the last incremental dump. The cost of performing a checkpoint in Multics increases 
linearly as the size of the system increases. To achieve the same degree of data security in a 
heavily used system where files are constantly created, destroyed and updated, as in a lightly 
used one, it is necessary that the backup system be invoked more frequently. This, in turn, 
implies degraded service to the user community. The basic reason why Multics (and other 
conventional centralized systems) are not able to provide Kill data security efficiently appears to 
be the inability on the part of the backup system to immediately discern when data has been 
created or altered and to reflect this fact onto the backup image of the system state. Because the 
mechanism by which data is created and updated is far removed from the file system with which 
the backup system interacts, the incremental dumping process is costly and inefficient. The end 
result is a computer system which cannot guarantee full data security without Incurring excessive 
Overhead. 

1.4.2 Distributed Systems 

Unlike single-processor machines or multi-processor machines under centralized control, it 
b difficult to perform global checkpoints in muffrprceessor systems i under distributed control 
because of the lack of any system wide synch ronizatwn capability. Hence, although distributed 
systems may have the potential of providing a more secure computing environment as a result of 
the redundancy present in their architecture {29], exptoiting such redundancy to achieve this end 
becomes much more difficult. A typical model charaeterkwg adiaribuusd system would be one 
where both the data as well as the code of a pirn-ess is spread oxer several physical nodes in the 
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system. Communication between processes in such a model is usually through some form of 
remote procedure call [25] or message passing mechanism 1 [2$ 18, 31J. FmIutc in these systems 
can cause processes to <feodyoci [2] after the recovery phase completes. To avoid deadlock 
problems arising from the error recovery of a process that is a member of a collection of 
communicating processes, local distributed systems or network computers usually have the 
amliry to set localized checkpoian^^^ 

points. If a &ikice of a process « detected, it is necessary tonofcbhly raft back the fanVdi>rocess 
to its most recent recovery point, but to alio reset all other processes that had exchanged 
information with the failed prooess since the ttotfof die last reeofefy point in order to avoid 
deadlocks from oosarring when operation s continued If these recovery points are not set 
property, it is quite likely to have * domino effect {2J whereby all* proiEessesmitfete recovery 
actions thai toad them to their earliest recovery rw*at Detenrtrririg whento set recovery points 
and finding a consistent set of such points is ancWtifvhfs process plf-ahd dfien stHjstfcntially 
reduces the overaU concurrency of the system. 

Borg etal[T\ and Barigazri {5} present ?«hemes fer ensuring the security of data in a 
distributed message-based system. The bask idea ia bodi proposes involves maintaining an 
inactive backim pioosat en a dirlemm ff a 

failure of a primary process occurs, the backup process ttkesovef execution, to Borg's scheme, 
messages exchanged between two processes are also automatically recorded on backup processes 
as well In addition, both the backup and recovery proesasea^e p^ocHeaHy synchrotiiied. 
When a failure occurs on a pnmary process, its backup begins execution in the state that me 
failed primary had at the tert syncbronwuion point T^e backup can eatch up to the state me 
primary had just before ttefeihj re by reconu^^ 

Barigazzi uses a similar approach in his system. However, instead of haying messages spnt 
from the primary to the backup process, an explicit recovery point is periodically established for 
every primary. Setting a recovery point for a process also requires establishing recovery points 
for all processes that communicated with this process since me last reajyery point was; set 
Recovery in this scheme involves rescuing all processes to their last recovery point While both 
these schemes provide full data security, they do so ^ me cost of high overhead , in maintaining 
up-to-date multiple copies of the primary process on disjoint processors. In addition, both 
proposals assume the transmission of messages to be a relatively inexpensive operation, an 
assumption which is certainly arguable. 
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In addition to general distributed systems, {here has been much interest of late in 
distributed transaction systems for which a high degree of data security is essential. A 
sophisticated approach to handling data security for transaction based applications can be found 
in the Argus system. The Argus integrated programming language and distributed system being 
developed at MIT [23] is a comprehensive attempt to pfOvMeJinguistie support for ensuring data 
security and consistency within a distributed competing iwrironment. The language provides 
constructs to encapsulate vital data objects which are then guaranteed to survive crashes of the 
host processor with high probability. In addition, a mechanism is provided to express atomicity 
of process activity. When a hardware failure occurs, each outstanding atomic action is forced to 
abort A two phase commit protocol is used to ensure mat all transactions preserve the 
consistency of the active data. To guarantee data security,, objects are distinguished as being 
either stable or volatile. All stable objects are written onto stable storage devices whenever a 
transaction completes. The integrity of stable objects -it* therefore, preserved even though 
crashes of the host node may take place. Volatile data is presumed to be lost upon failure. 

Perhaps the greatest point of contrast between the Argus implementation of data security 
and that which we want to provide in our system is-the requirement in Argus mat the user 
prespecify those objects which are to survive crashes. Our goal is to ensure that measures taken 
to provide data security are transparent to the user; jk> linguistic primitives are provided to 
specify the objects he wishes to survive failure and no expfeit constructs are provided to control 
backup operations. While the gap between the programming model and the backup facility is 
not so severe in Argus as in Multks, the feet that the underlying system is based on an updatable 
memory execution model makes the task of ensuring a consistent state a complicated one 
involving expensive protocols such as two-phase commit and sophisticated algorithms to 
maintain a correct and up-to-date backup image. Moreover, the application domain for which 
Argus is best suited is a relatively restrictive one and the complexity of the system is probably not 
warranted for most non-transaction based computations. 

1.43 Fault Tolerant Systems 

There have been several proposals put forth for the design of fault-tolerant data flow 
computers which attempt to achieve data security (as well as availability) by incorporating low 
level fault masking capability into the system. We outline several of these proposals here to 
present an alternative approach to solving the problem of providing data security for a computer 
system. 
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An error recovery system in the context of a fault-tolerant data flow machine based on the 
Dennis-Misunas architecture |9J was described by Mtsunas iflf24]P -The model advocated was 
based on providing triple modular redundancy (TMR) for memory and the ftmctidriat itnits. 
Each instruction cell acts as a voter and reeei ves three results lor^eaeh operand generating error 
packets if discrepancies are found. In addition, pnteess^failttW#£h&^ the 

network to be nsconfigared. The scheme involve extensive overhead in t&ms of increased 
packet traffic and extra hardware and is dependent oe the *b!e of acknowledgment signals to 
allow reconfiguration to Replace. A rault-tolcra«t design for aStade'dAtaflbw^rehitectare was 
also proposed by Leung {22}. In his proposal, adyaaHite feiftihaineV technique was employed 
whereby redundant units are used to detect and dfegnote feufte, wfth afflicted corriputarlons 
reexecuted when necessary. ;jf 

Hughes (19} suggests that a checkpointing strategy suftaMy modified for data flow 
computadoa may bean appropriate mechanism for incorptfi«mg iflor recovery (n a data flow 
machine. Basically, each cell <or instruction-) which is checfcpomted is marked. Whenever the 
backup system initiates aoheekpomt, alt active celte^flievioi^ehelfcpd On 

recovery, all active cells that were checkpointed are restored. While simple to implement, 
recovery in this scheme requires the reinitialization of the entire state, which is an often 
unnecessary step. In addition, the memory of the system must be examined at each checkpoint 
to determine which items have not yet been copied; this is also quite wasteful in most instances. 
Finally, this proposal assumes that all information is resident in memory whenever the 
checkpoint process i& invoked; by contrast, in die system- whic* %e envisage, data wflf be spread 
across the memory hierarchy between disk and mairrstore du r^ pr ogr am execution. 

AH three of the proposals cited here are based on an architectural model much different 
from the dynamic data flow model designed for Vim. Moreover, , they require special 
architectural enchancements to mask hardware faults. Vim is not intended for applications 
which have high availability requirements and, thus, there is no need for fault-masking strategies 
to be incorporated into the system. Consequently, &m appre&ches to »ac#iie^mg data security 
taken by these schemes will not be directly applicable m satisfying the requirements we have set 
forth for our system. 
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1.5 Thesis Outline 

In the next chapter* we present a detailed model of the Vim Computer System. We 
describe in detail the applicative base language being used amtHie dynamic data flow execution 
model employed. In addition, a formal semantics of system behavtoor is also developed to 
rigorously define its operation. The manner in whicte users communicate with the system and 
the means by which tong-iived objects are defined are also presented. To simplify the 
presentation, we assume a system supporting only a single user. Because of die applicative 
programming model used in Vim, this simplification does not invalidate its applicability to a 
multi-user system. We do not consider non-determinate computation in our model. This 
restriction also simplifies the algorithms. The complications involved in incorporating non- 
determinacy into the model is a topic which we discuss in Chapter Sk. 

The third chapter in the thesis presents the general strategy for the backup and recovery 
system. We define die criteria which the backup system will use in determining what 
information should be recorded on the backup storage mediuaL We also present the 
architectural enhancements necessary to support the backup and recovery utiikies. 

In the fourth chapter, we give a more detailed description of the backup and recovery 
algorithms. The alterations necessary to the interpreter are discussed. We also formalize the role 
of the backup and recovery system in the context of the abstract model given in Chapter Two 
and present the changes necessary to the base language mstructions to support our algorithms. 

The fifth chapter describes low level details » the transfer of information between the 
backup storage medium and ihe Vim system. We discuss bow tr guarantee the atomicity of 
information transfer so that a consistent image of the system state is maintained on the backup 
storage medium. We also mention.how storage management is handled on the backup store as 
well as describing how to minimize the copying of information from system memory onto 
backup memory. 

The final chapter presents a summary of the thesis and discusses some topics for further 
research. We pay particular attention to the changes which may need to be made to the backup 
and recovery algorithms if our model of system behaviour is augmented to aHow non- 
determinate computation. 
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Chapter Twa 
The Vim System 

The Vim Computer System is an experimental project in the Computation Structures 
Group intended for the 'investigation of a novel data flow architecture for the efficient support of 
runctional languages. In this chapter, we discuss the design of the system and present an 
operational semantics for its applicative base language. The main programming language 
supported by VIM is VimVal, a major extension of the applicative language Val [\]. VimVal 
programs are translated into the base language data flow graph representation which is then 
executed by the Vim Int<rpm*r* Users communicate ftttti .iViMr ttaougb/a iSAeflprogram. The 
Shell provides apQwerfiil inter&ce to the system that atow$ users to build and manipulate Vim 
environments - the main ifcejhty for constructing k^Ui^suurturcs. We discuss each of these 
components, VimVal. the Interpreter, and the Sn«tt»ilw/otowng sections. 

2.1 The Applicative Language, VimVal 

In this section, we present an informal overview of the Vim high-level applications 
language, VimVal. VimVal differs fee** Val, its? predecessor, in that functions m treated as 
first-class objects,! fteeuse^iecursion and mutual recurskin is alkjvwi and polymorphism is 
supported through a Milner-style type inference mechanism. Users can program tea structured 
and hierarchical manner through the use of a /rrail»^constn« described below. 

In addition to various scalar types such as integers, reals, characters and booleans, VimVal 
also contains structure types such as arrays and records. Users can express history sensitive 
computation through the use of streams{3Q]. A stream is a potentially infinite sequence of 
homogeneous values which may be of any type (including stream) that allows users to write 
history sensitive code (such as the modeling of a conventional memory cell) within a functional 
framework. There are three operations on streams: first which returns the first element of a 
stream, rest which given a stream returns the stream withWt its first element, and affix which 
affixes an element to the head of a stream. We shall discuss the implementation of streams in 
greater detail in the next section. 

Because functions are treated as firsticlass objects, they may be passed as arguments to 
other functions, returned as the result of a function activation and may be built into data 
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structures e.g, array[Function]. The body of a function is an expression whose type is the result 
type of the function. The form of an expression may contain conditional expressions, function 
invocations, tagcase expressions which aMOw one of* series of expressions to be chosen based on 
the value of a tag, and let expressions which are viewed as sugaring for lambda abstractions. 
Conventional iteration in VimVal is expressed using tail recursion. Variables in VimVal are 
considered as only identifiers for a value. Once an identifier is defined, it cannot be changed — 
VimVal is a single-assignment language. The treatment of variables as identifiers for values as 
opposed to placeholders for memory cells which may be arbitraijty mutated as found in more 
conventional constructive languages is a distinguishing characteristic of VimVal, 

A module in VimVal is a function which may be itwcked firom other modules or by a user 
command to the system. A module provides a «ect*atiisrii for grouping related functions 
together. These functions may be invoked from wfcttin the module or, if they are passed as a 
result of the module, by functkws externa* to th* module, Thebbdy of a module may use names 
that are not bound by definitions in the module These free names must be bound before the 
module can be run. Modules may be separately compiled with 8 type consistency of identifiers 
used across modules being checked by a linker. Type specifications are optional in VimVal. 
Omitting type definitions allows free names in module* to be bound in possibly several contexts, 
each binding causing a different resolution of the types of the free variables. Users can, thus, 
express useful polymorphic functions using the type inference mechanism. For a more detailed 
exposition on the VimVal language, the reader should see [13] and (28J. 

2.2 The Vim Interpreter 

The base language for the Vim system is the language of dynamic data flow graphs. 
Programs written in VimVal are translated into their data flow graph representation. Base 
language programs are evaluated by the Vim interpreter. A dynamic data flow graph is a 
directed acyclic graph in which nodes represent base language instructions and arcs are used to 
indicate data dependencies among instructions. There are two types of arcs inthe graph: value 
arcs and signal arcs. An arc (s. /) is a value arc if it carries the value produced by the execution of 
node s to node /. A signal arc is used for performing a control function such as selecting which 
arm of a conditional expression is to be evaluated. A node is said to be enabled if and only if it 
has received all necessary values and signals. Enabled instructions can be executed in any order 
and the interpreter is free to choose the execution of any cnaWcd instruction from any current 
function activation. 



§2.2 THE VIM INTERPRETER 



13 



To illustrate the structure of a Vim data flow graph, we show in Fig. 2 the base language 
representation of the VimVal function shown in Fig. 1. Instructions are drawn as boxes, with 
the opcode of the instruction labeled inside the box. Value arcs are drawn from the bottom of 
the instruction which produces the value to the appropriate operand slot in the target instruction. 
Thus, operand number one for an instruction will be sent along the leftmost value arc entering 
that instruction. Signal arcs are drawn catering into the side of instructions. Each instruction has 
an index in the activation used for addressing purposes. The behaviour of! the base language 
instructions are described in greater detail later in this chapter. 



Function /(x :boolean; h, g : Function returns int) 
if* = true 

then HI) % A returns an integer, 
else £(2) %g returns an integer. 
endif 
endfun 

Figure 1: A simple Vim Val program 



Any non-scalar value produced during the evaluation ofan activation is placed on the Vim 
heap. The objects which may be found on the ircap include function templates and data 
structures such as arrays and records. Associated whft every! object is a unique identifier, uid, 
which distinguishes this object from every other objed on th^heap. Thus, unlike simple scalar 
values, data structures and function templates are Hot trafSmitted along value arcs in an 
activation. Rather, the result of producing a complex ^ stnictur^^tp flacj this. strucJwre 1 on the 
heap and to send its uid to all target- d est in a tio ns instead, - Vim e mp l o ys a reference count 
mechanism to manage the heap. When there exists r^refci^^a^^^ from 

within any current activation, that, structure can be re^^dj^tfie heap and Us space reused. 
The Vim heap may be considered to be a multi-moled, dire^e^^gl^gr^ph where an arc (s, t), 
connecting nodes s and t, indites that f is a cofnponen|Uof 4: , Elements on the Ijeap may be 
safely shared among objects. This feature is a result of the aflpjjcativf! nature of the base 
language. An object on the heap consists of a unique identifier and a structure value. 

A distinctive feature of Vim base language programs is that no arc in the graph is ever 
reused. This is a consequence of the graph being acyclic with tail recursion used to model 
iteration. This model of data flow graphs requires that every new function application create a 



-^m<i!!^^^~^m^-i^^^m^^^^^^. 
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dl' "dk 

/. Before APPLY* executed 




dl' "dk 

Z Aft* APPLY extent*. 




3. AJier activation completes with return result*. 
Figure 3: Function application in Vm 



2 J The Vim Shell 

Users communicate with the Vim system through a system. Shell. The Vim shell is 
responsible for accepting user commands, andf translating them into the appropriate base 
language representation and then invoking the in terp reter t o e xecute this base language program 
whenever necessary. A user session typically consists of the iiser communicating with the Shell 
in an interactive mode, inputing Shell commands whose results are subsequently output to the 
user. Every user executes in a unique environment. A VlM^vfronmelt relates symbolic names 
to values and actsas-a repository for all long-lived objects; O b j e c t s refer e nced in an environment 
must be explicitly deleted by the user. The Vim environment plays the role of a directory 
structure in conventional systems. Users specify that a particular <name. value> pair is to be 
placed in his environment ft^ 



jfl 



bind name : = <expression> 



„> * **&*„ "*--» *-* ! 
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binds the value of the expression to the name specified: This binding is then placed in the user's 
environment In addition to the bind command, there is a DELETE command which, when given 
a name, removes the <name, Value> binding from the user's environment The command: 

DELETE^ 

removes the binding, <N, Value> from the current .envifattifl&iL Uiers must execute a delete 
command to have an entry in their Vim environment removed 

The Vim shell translates a bind command into its base language representation which can 
then be executed by the Vim Interpreter. The translation for the command bind x := f(z) is 
shown in Fig. 4. The APPLY instruction creates an activation of the function /with argument z. 
The result of this activation is sent to a special instruction, TERMINATE, which informs the shell 
that the value which is to be bound to the name ^ is available. The storage occupied by this top 
level activation can then be reclaimed by the systej^\jsuig the RELEASE instruction. Thus, once 
the value which is to be bound to the symbolfcoame is kho^n, the activation created by the shell 
can be removed from the system. 
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Figure 4: The translation of the bind command 



RELEASE 



% m. v i * i'*»«l 'i * ijM « '' 



There is no translation into a base language program necessary for Jhe DELETE command. This 
command is processed by the shell entirely — no assistance is required from the Interpreter to 
execute it 
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The terminate instruction described above is tised to synchronize the operation of the 
interpreter program with the shell: If this synchronization mechanism were removed and bind 
commands were allowed to arbitrarily Overlap with one another; it is possible for incon^ 
environments to be constructed. To see why, consider two BIND commands input in succession: 
BiNDfo F) and bindU G). It is clear jhat if the con^nds^re-j^cs^ x should 

finally get bound to the result of evaluating £ Jo ensure, that a thJsjB^iaIization takes place, 
however, requires mavmesecc^Bl^p command does no* ojjcuj; until the Mist onecsppfctes. 
As we shall see later, it is possible to still exploit a gj^ania*ntj)fj5p^cu^gycy by,aHowM>»the 
computation of Fto be still proceeding even if BIND (*.# executes. This feature, known as early 
completion* is described in section 242. 

In the next section, we will present a formal operaticmaTinoW for the VlM interpreter, 
base language and shell. This model, called Ml, will jjef ecessarily vjer# abstract We wjlj not be 
considering, for example, internal representation of data strucnires in memory, paging of 
structures to and from memory or scheduling algorithms. We make two simplifications of the 
actual system in our model. First, we assume that all coniftjtatRkf in Me system is determinate. 
A computation is determinate if its output is totally specified by ih^, ; v#lue of it* inputs; it's 
output does not depend on such factors as the relative arrival time of its inputs. Secondly, we 
assume that only a single environment exists in the system and, thus, there ts no need for 
providing an explicit environment name to those instructions which manipalate environments. 
In the following chapters, we shall refine this model tq describe the backup and recovery 
algorithms. 

2.4 A Formal Operational Model • Ml 

The Vim system contains three major components: an Interpreter, a system State and a 
Shell. The State embodies all current information in the system L&, heap, activations, enabled 
instructions and environments* The interpreter e*ecyjes a base iangu^e representation of a 
Shell command and returns the value of that comimnd* iThe^ahieieiuFnedhytiie interpreter is 
bound to a symbolic name in the user environment The name feeing bound is determined by 
the current bind command being processed. A shell command is translated into its base 
language representation and is then executed by the interpreter, Thisitianslation is performed by 
the Shell. The shell as described above is a function mapping fuom a Smteand a session to a new 
State. A session denotes the history of SVit'// commands input to the system. The shell translates 
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shell commands, invoices the interpreter to execute these, commands in the current state, and 
binds the result of these commands to names in the use* environment It returns the state which 
is produced after evaluating all shell commands in thesession. The abstract architecture of the 
Vim system is shown in Fig. 5. 

In the following discussion, sets are denoted by bold font, elements of sets are denoted by 
italicized letters and names are indicated by a script strings of tetters, thus, Set is a set, Ell € Set 
and tag is a tag. We present our semantic definitions usIhgifofVAL-iike syntax augmented with 
operations for performing set abstraction, set membership, etc. ori the domains defined below. 
Function domains are specified using arrow (^f) rotation, thus, tile domain equation, 
A - B -► C defines A to be the set of all functions with cferrMh B and range C. Tuples are 
enclosed using angle brackets. 

Formally, we define the Vim System to be a three-tuple: 
ViM = <Shell, Interp, State> where 

State f <Act X H X EIS X Env> 

Act a U A -♦ Activity 

H = U H - ST 

U A = the set of unique identifiers used for activations. 

U H = the set of unique identifiers used for structures. 

EIS = the set of enabled instructions, described later. 

Env = Name — (U U Scalar) 

Activity -N-* Instruction. N being the set of natural numbers. 

An element Act m the domain of Vim activations, A<fc is a mapping from unique identifiers to 
activities. An activity is a function mapping from natural numbers to instructions and represents 
the code of an activation. An activity can be thought of an array of Instructions, the f h element 
in the array specifying the t h instruction. The Vim neap, H, is modeled as a function from 
unique identifiers to structure types, ST defined below. The stricture of the heap is determined 
from the mapping defined by the heap function. Scalar values are not represented On the heap. 
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Figure 5: The Abstract Vim System 



In the actual system, such values are transmitted directly along the value arcs in the graph,. The 
Environment component in the State is also a function, mapping from a name, which is any 
sequence of characters, to either a unique identifier referencing a structure on the heap or a 
scalar value. 

The data types supported by the system are given below: 
Scalar* = Integers U RmJs JL> Bootews U Character UNuJI 

Name = Character*, the set of aH character sequences! 

Integers = the set of all integers K)\undefi 

Reaffe ; = the set of all reals U {undef) 

Rooteans =± {mm false, undej) 

Character = the set of characters in the machine U {unde/} 



Null = {nil unde/} 
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ST = {ArrayU{w/u/e/}} U {RecordU {undefi} U {QneotUiundej}} U {Function} 
U {ECQ} U {Clsr} U {Dests} 

Array = Z -* (U H U Scalars), Z being the set of integers. 

Record = N -» (U H U Scalars U SUSP U Dest), N beingthe set of natural numbers. 

Oneof = N -» (U H U Scalars U SUSP), N being the sefcof natural numbers. 

Function = N -♦ Instruction, N being the set of natural numbers. 

The set of structure types includes arrays, records, and oneofs. Arrays are modeled as functions 
from integers to either unique identifiers representing structures on the heap or scalar values. 
Records are modeled as functions from natural numbers to either unique identifers representing 
some structure on the heap, scalar values, suspensions, which we describe later, or destination 
lists. As we explain below, the return address of an activation is packaged into a record and 
transmitted to that activation when it is instantiated by the apply instruction. The destination 
list represents the list of return addresses which are to receive tie result of the activation. While 
components of record structures are addressed by their fieid name in VimVal, the compiler 
translates these names to the offset of the addressed component in the record 

Note also that the set of functions is also included among the elements of the structure 
types in the system. This is consonant with our treatment of functions as first class citizens. An 
element of type Function is a mapping from natural numbers to instructions just as elements of 
the set of activations are. As we shall see below, the only difference between a function 
definition and its corresponding activation is that the latter is sensitive to the effect of instruction 
execution since operands and signals are received by the instructions within an activation 
whereas a function is a pristine object like any other Vim structure. 

A function closure is a special record of two components: the first component is the utd of 
the function template of the closure and the second component is the h>t of free variable 
definitions found in the function. The closure of a function completely defines the bindings of 
the free variables in the function and, thus, must be-dctiri^';b^6^^tuiitibn can be applied. 
Free variables are accessed by its index in the free variable H$t The definition of a closure is 
given formally as: 

Clsr = <U„ X (N - (L,, U Scalar))> 
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2.4.1 The Base Language Instructions 

A base language instruction is an seven-tuple consisting of an opcode, three operand fields 
(not all need be used), an operand count used to indicate how maity operands must arrive, a 
signal count used to indicate how many signals «ust*erieeived, and the destination record 
containing the list of desiinaaons fbr this instruction. The set of opcodes we will be considering 
in our operational model will iiictode reco^;^ application 

instructions, and operations as early completion dements, c These classes of iristrueGons wiH be 
the ones of greatest interest when we present our backup and recovery algorithms. 

Instruction = OPS X (U H U Scalar*) 3 X N X N X Dests 

A destination has a type which is either unconditional if the result of the instruction is to be 
sent automaUcally to the destination, true or false useAJ^raSMiCMirBBmctioh to control 
conditional evaluation. If the type of the destination is true, then the result is a signal which is 
sent to the destination instfucttoti it and only it the SwtrcM operator evaluated to true A similar 
description apiihes for a false type destination. All destinations of an instruction must be within 
the same activation, the second component ofa destimu^nVtte 

the signal or result value should be sent If the destination k to receive a signal, men this must 
be specified. Otherwise, the operand field to which the result is to be sent must be provided 
For convenience we shall refer to the elements of an instruction using dot, "? notation. For 
example, the opcode of instruction /shall be denoted as I.opcode etc 

Dests = 9(D) 

D = {uncrjrta, true, false} XNx {opt, op2, op3, sianal} 

An enabled instruction is a two-tuple («, i) representing ae instniction in some activadon which 
has received all necessary operands .and signals. Any enabled instruction can be executed by the 
interpreter. The applicative nature of the system guarantees thaMh* fcellavlotrf af theprogram 
wiU be determinate regardless of the order in whickeoabJcdr instructions fcisthe program are 
executed. 

EI = <L1 A XN>. 

EIS = 9(EI) is the set of enabled instructions. 
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2.4.2 Early Completion Structures 

According to the model of operation presented above, an instruction is allowed to execute 
only when it receives all necessary operands and signals. In the case when an instruction is to 
operate on a data structure such as an array or recocd, this means that the entire structure must 
be fully constructed before the instruction can execute. If the instruction only needs to examine 
a certain portion of the structure, then the execution model unnecessarily constrains parallelism 
in the program. To alleviate this problem, there is a facility in Vna known as early completion. 
Early completion structures allow greater concurrency of operation by allowing a data structure 
to be constructed and used before all of its components are ayjulable. , The compiler designer^an 
use this facility, for example, to generate code which will cause the results of an activation to be 
an early completion structure to allow the calling activation to use some Of the results of the 
callee before all of them are known. 

An element of the set ECQ is an early completion element [11]. An early completion 
element is a two tuple, (u, /), where u € U A and / € N. The ea/ly completion stnjeture is 
essentially a queue containing target addresses Qf those instiuctions which require the value of 
this element in the structure. When the value is finally produced, it will replace the ec-structure 
and will be sent to all targets. This process is illustrated in Fig. 6. 

ECE = <U A XN>. 
ECQ = 9(ECE) 

An element, (u, i) € ECE denotes an instruction which has requested the value which is to 
repine this early completion structure. The uid u denotes a function activation, and / is the 
mdex in mis activation of the target instruction. 

143 Delayed Evaluation Using Streams 

The astute reader would have noticed that the stream type presented in an earlier section is 
not defined in our formal model. We represent streams using the record and oneof types: 

StrcamTT] = oneof 
[empty : null 
non-empty : record 
[first : T 
rest : Strcamf fUJ 
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Figure 6: The Use of Early Completion Structural 



After B executes, value of R.k is sent to all targets. 



We have previously mentioned that streams ar9 r ^p^^^.|i|Qjtltte- ^iucture». In a purely 
data driven execution model, the production of a stream may far outstrip its consumption. To 
avoid wasteful computation, streams are produced in a ^m*^<fov<?fl fashion [16J. in demand 
driven evaluation, an element of a stream is produced only when i|s consumer requires it To 
implement this feature, a special record element called a,m#ens$^ is introduced. A suspension 
contains the address of the instruction in the stream producer responsible for instantiating 
production of the next stream element When a consumer access a^ suspension, die suspension 
becomes replaced by an early completion queue and a signal is, sent to the address which the 
suspension holds. A new record is created for the next element and the early completion queue 
will get replaced with (he uid of this record. The head of this new record will contain the new 
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stream element and the tail of this record will again hold a suspension. We illustrate this process 
graphically in Fig. 7. 



I. Stream S with first(S) = v. 
S 



2. After suspension accessed, it becomes an ECQ. 
S 



Susp 
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ECQ (u.i) 



Producer 



(u,i)isthe consumer instruction 




3. Producer yields next element which is transmitted to waiting consumers. 



Figure 7: Demand driven evaluation of a stream using suspensions 
SUSP = <U X N> 

The suspension structure is a pair <u, i> which represents the address in the stream 
producer that is to be signalled when the suspension is accessed. 

15 Semantic Functions for Ml 

In this section, we present an operational model for the Vim Shell Interpreter and base 
language instructions which extends the operational model defined in the previous section. The 
instructions which we examine here are chosen because of their relevance to the backup and 
recovery algorithms devetoped in the following chapters. We partition our presentation into 
four categories: formal description of the Shell and Interpreter, instructions which operate on 
structure types, instructions which are used for manipulating early completion structures and 
stream elements, and instructions which are concerned with function application and return. 



§2.5 SEMANTIC FUNCTIONS FOR Ml 



25 



In the next section, we define sortie aoxlliiiy functions that will be useful for our 
presentation. These functions are used In the <fcH^ base language 

instructions. ' 

23.1 Auxiliary Functions 

There are several primitive auxiliary functions whteh w£iteed to define before presenting 
our operational model l^iese functions operate ton tfte^ftelsp, aetivati&n, and environment 
components of the Vim state. The functions 'fl&M&ft *^v*&#adds and remove an 
element to and from the heap respectively. ThelVewfi^lum^ takes in thiee arguments: a 
current heap H, a uid u, and a structure, v. It returns a new heap, H *, identical to H except that 
this new heap is defined for u such &m H*(ttf = * TTii^cfefirnldh of NewHeap allows us to 
rebind existing uid's to different values, a feature which, will be usefol whea implementing early 
completion structures and suspensions as we shall see later. The Remove Heap function takes as 
arguments a heap H and uid u, and returns a new heap, /TidenticW to\# except that H\u) is 
undefined. 



NewHeap: HxU H XST-»H 

Function NewHeap (H, u, v) 

V Ul € U H let H\u$* mM* «i * u 

= vothetwtee 
in 

H' 
endlet 
endfun 

RemoveHeap: HXU H ->H 
Function RemoveHeap (H, u) 

V u, € U H let //*(Wj) = H( Ul ) if u x * u 

= undefined otherwise 
in 

H' 
endlet 
endfun 



There are similar functions, AddAct and* Remove Act defined for the activation component 
of the state and AddEnv and Rcmovellnv defined for the environment component as well. 
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Since the transmission of results and signals is, an acuity cominon to every instruction, we 
define an auxiliary function, SendToDest which is responsible, ,fpf pending a value or a signal as 
the case may be to the specified target instruction and constructing a new activation function and 
new set of enabled instructions to reflect the effect of this transmission. SendToDest constructs a 
new activation component in which the target instruction found in die destination argument has 
been updated to reflect the transmission pIF^^^^^^^^mm^^-^^ transmission, 
the target instruction becomes enabJed4&, have its opcnt and silent fields become zero, that 
instruction is appended to the enabl^MwcjipttSfk 

ResultType = (U H U Scalar) U signal 

SendToDest: Act X EIS X U A X D X ResuttType -* Act X EIS 

Function SendToDest Act, EfS, u FA , <*, i, opnum>, /wo/0 

ktFA = Act(u F J 

resultvai - if result - signed, 
then undtf 
eke result 
endtf 

newopnuml = if opnum * opi then J &pl && resultvai 
newopnum2 = if opnum * op2 then /.Offc elst/wu/fwp/ 
newopnumS = if opnum * op3 then /.op3 else resultvai 

newopent = if result = signal 
then /.opcnt 
else /.opcnt-1 
endif 
newsigcnt= it result = signal 
then /.sigcnt-1 
else /.sigent 
endif 

/'= /.opcode X newopl X newop2 X newop3 X newopent X newsigent X /.dests 

Vy€IN/vT(/) = FA(j) t j*i. 

= ''. J=i. 

New Act - AddAct{Act. u FA . FA")% new set of activations 
Newl-lis = if {/ .opcnt = 0) A ( / .sigpnt ..=* Q) 
then EJS'u'i<u F/t . Of 
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ekeEIS 
endif 
in 

NewAct, NewEis 
endfun. 
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Two functions which call SenaToDest are . SendYotye and SendSignal. The function 
SendValue calls SenaToDest for every target in the destination list of an instruction whose opnum 
is not signal That is, all target instructions that are to receive^e result of the instruction are 
sent the result value by SendValue. This operation is accomplished through the use of 
SendToDest. SendSignal operates in a similar feshicmPw SlJrttfJ&fo* exempt tfrat it calls 
SenaToDest for all targets in the destination list of an taftrucjion whose,o/^«ffi is signal. 



SendValue: Act X EIS X U A X Dests X ResuhType - Act X CIS 

Function SendValue (Act, EIS, u FA , dests, v) 

let ValDest = {<de, i, opnunO € dests \ opnum € {opl, op2, Ctp3}} 

opnumj> <dc n , i^ opnum ^ = 

n components of ValDest 
in 
SenaToDest( 
SenaToDest^*. 
SendToDest(Act, EIS, u FA , <dc v i^opmtmf, ») 

u fai <^t^opnw^>, v)„j , 
endlet 

endfun 



SendSignak Act X EIS X V A X Dests X ^ Act X EIS 

Function SendSignal (Act, EIS, u FA , dests) 

let S/gZ)«/ = {<<&,/, opnum>£ dests | cp/»wn = signal} 
<dcj, /,. opnm^.<dc 2 ^opmm^,..., <<fc^ / m , opnum ^ = 
m components of SigDest 
in 

SendToDest( 
SendToDest(... 
SendToDest(Act. EIS. u FA . <dc v /,. opnum x >. Signal), 
u FA . <dc r i r opnum 2 >. signal)...) 
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endlet 
endfun 



15.2 A Formal Model of the Shell 

As we described above, the Vim shell serves as the interface between the interpreter and 
the Vim user. The formal definition of the shell is given below: 



Command = C X Name X (Exp U undej) 

C = {BIND, DELETE} 



Session: StreamfCoBunand] 
Shell: Session X State - State 



Function Shell (Session, State) 

let <Act, H, EIS, Env> = State 
NewState = 

WChooteToExecute (State, Session) 

then ShelKSession, Execute (State, Choose(EIS))) 
ekeifemptrf&ttio/]) 
then State 

else let c, = tirst(Session) 
in if c r C = DELETE 

then <Act, H, EIS, DelEnv(Env, Namep 
elsetf Cj.C = &ND 
then let %commafi<£is SKA) 

FA = TranslateCCj) 
u FA = new uid front U A 
Act' - AddAct^Act, u^ FA) 
NewEIS = ElSX)i^u FA , t> [FAi^affcnt - 
A /V4(i).sigcrtt = 8} 

State', v = Interp(State, Choic4NewElS)) 
<Act\ H\ £75". Emi> k State' 
Enr = AddtoEnV{Entc v iUtrtK, v) 
in 

<Act\H',E/S\Env-> 
cndlct 
endif 
endlet 
cndif 



>-. T^r^^t* jj*?*^ -& "r- *?# <** ■ 
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in 
if empty (Session) 
thtnHEfS'*{} 

thenShelHSestion,Execute(Newstat£,ChotaiEIS'))) 
else Newsiate 
endif 
else Shell (nst(Session), Nemtate) 
endif 

endfet 
endfun 



The shell takes in as input a s^wim of sh^PeOiWhiinds: J It Cans the function 
ChooseToExecute which examines the current state and fcttten and determines if the shell 
should process the next shell command or whether it should call \ht Execute function using the 
current enabled mstraatorr Set This function allows "ffies^tem to continue to process 
instructions even if the remainder of the session has noijetiJtwipputily the user. One possible 
implementation of this function would be a routine which examines die current input buffer — 
if the buffer is empty and there are instructions still to be executed, it returns true. If, on the 
other hand, there are shell commands available to be processed, it returns false. In die case when 
there are both commands and enabled instructions avaitable, it^can aj^r^rily fe|urn either true 
or false. If the function returns false and the first cortrnasd »4hecommand stream is delete, 
then the shell removes the <name, value> bmdir^fr6rrfthe culrW^envirohmelit and processes 
the rest of the command stream. If it is a bind comnw^ frwwef. it calk an auxiliary function, 
Translate with this command stream element Tran sla te r e ftir i* * an activity, ffi which embodies 
this command. For example, if the command input to Translate was bind.^ = f(z)\ the 
activity returned would be of the form shown in Fig. 4. The result of evaluathifclSris activity 
represents the value of the command. A new activation is constructed from this activity and this 
activation augments the current activation state. The enabled instruction component is also 
appropriately augnientedt&irK*^ 

and signal count already ien>. The shcW ealfcthe 4nferpr*ter *w1ifll*1lnfene# Slate and some 
enabled instruction from the set of cabled ihS^^ which enabled 

instruction to execute is made? by *e Gtiolee functSah. f fHe>na^*aie if^tftrhetJ' by the mtCrprefer 
is used by the shell ^processing the next shell comnrrtnd if ihere arc any more to b^ processed. 
Ibe value, v. rearmed by the internfctier ir&Ound in Hicmer ehvh^Hrnent to the symbolic name 
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which is an argument to the bind command. If there are no more stream commands to be 
processed, the shell calls the Execute function described below to execute the remaining 
instructions found in enabled instruction set If there are no more enabled instructions, the 
function returns with the final state. 

U53 A Formal Model of the Interpreter 

The interpreter is a state transition function from states and enabled instructions to a state 
and either a unique id or a scalar value: 

Interp(<Act, H, EIS, Env>, Choice(EIS)) H- <Act\ H\ EIS', Env> X (U H U Scalar) 
The Choke function, as we explained above, is used to determine which enabled instruction 
should be chosen for execute from the enabled instruction set The definition of the 
Interpreter is given below: 

' IN ■,..—■ — M i l I II ■■! ! I |! I,, || ,| J , 11,111,1. ,|| ^ | . M M pm 11,11 m i m i .. n il. ■ ■■■■■ „ ■ I ■ 

Interp: State X EI -> State X (U H U Scalar) 

Function Interp (State, <u, i>) %<u, /> is an enabled instruction 

let 

<Act, H, EIS, Env> - State 
FA - Aciu) 

Newstate = Extcutt(Slate, <u, L>) 
<Act', H\ EIS', Env> = Newstate 
in 

if FA(l}.opeOOM * TERMINATE 
then Newstate, FA(i).opnumi 
else Interp(Newstate, Choice(EIS')) 
endif 
eadlet 
endfun 

The instruction which is chosen for execution must be part of some activation defined in 
the set of current activations. Act. The interpreter calls on an auxiliary function, Execute, 
defined below which contains the definitkmsof all the base language primitives. Note that the 
interpreter only returns its result when the terminate aistcuction teas executed. This restriction 
guarantees that environments will be updated correctly according to the order in which bind 
commands were input. When the interpreter returns, a new command can be processed by the 
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shell. Note that because of the presence of early completion structures, there may still be many 
activities m progress at the time the terminate instruction executes and the next shell 
command is processed. Thus, our model allows instructions found in activities created from the 
evaluation of different bind commands to execute in parallel^ We do not come into problems in 
augmenting environments though, because, as we discussed earlier, bindings are always 
constructed in the proper serial order. 

The Execute function examines the instruction feeing processed aid performs the necessary 
fijnctfcm. The result of this fenctkm wMt be a n*w Vallate. Thestructure of this function can 
be given as follows: 



Execute. State X EI -♦ State 

Function Execute (State, <u FA ,k FA >) 

let <Act, H, EIS, Env> = State 
FA = Act(u FA ) 
I = FA(k FA ) 
destinations = /.Dest, 

NewEIS = EIS - {<u FA , k fA >} 
in 
if /.opcode =s set then... 
ebeif /.opcode = apply then ... 



endif 
endlet 
endfun 



The destination list specifies those instructions in the current activation to which the result 
of executing this particular instruction should be sent. Recall that in addition to transmitting 
results, we may also need to send signals to destinations. 
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2.5.4 Formal Definition of Base Language Instructions 

In this section, we present a formal definition of those base language, instructions that will 
be useful to us in describing the backup and recovery algorithms later in the thesis. Keep in 
mind that these definitions are actually found within the Execute function given above. 

2^.4.1 The terminate Instruction 

The terminate instruction is used to receive the result of evaluating a base language 
program. This result, value is then bound by the shell in the *uer environment The instruction 
takes in one argument, which is either a scalar value ©r a uid it sends nonsuits but will send a 
signal to a release instruction which is used to remove the activity from the set of current 
activities. We describe the operation of the release instruction later in the chapter. The 
interpreter picks up the result from the first operand slot in the instruction when it returns back 
to the shell. 



if /.opcode = terminate then 
let 
val = #(/.opl) 

Act', NewEis' = SendSignaKAct, EIS, u FA , destinations) 

in 

<Act',H, NewEist Env> 

endlet 
endif 



15.4.2 Structure Operations 

The base language contains powerful instructions for the creation and manipulation of 
structure types. There are three structure operations of particular interest - create, REPLACE 
and SELECT. The create operator is used to create a structure of a particular dimension. In a 
functional language, a structure, once defined, cannot be subsequently altered. Replacement of 
an element a in a structure with an element ft is done by creating a new version of the structure 
with (} replacing a in the new version. The select operation returns the value of a specified 
field in a structure. 

The operations we describe below are for the record structure type but are very easily 
converted lor the array or oncof type. 
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To create a record structure, we have a makerec instruction. It takes in one argument, the 
size of the record structure to be created and constructs a record of such a dimension, setting all 
the fields in the record to be undef. In addition to the MAKEREC iastruetion, there is also a 
makerecec instruction which constructs a record, an of whose elements are early completion 
structures. 

■ "■ ■ . » 'I 'i ■» i , t I ■ U i .■ » r i. ii , 

if /.opcode = makerec then 
let 

m = /.opl % m is a natural number 

« y = a new uid in U H , 

Act', NewEis' = SendValue(Act, EIS, u- A , destinations, u J 
Act", NewEis" - SendSignaKAct', NewEis', u^ destinations) 
H = NewHeap(H, u^, MakeRecord{l,m,tmd$fi) : 
Ui 
<Act", H'. NewEis", Env> 

endlet 
endif 



The replace instruction on records takes in three argumwt*, the uid of a record R, the 
field m me record which is to be repMedjf and the value!pfi&ijnew element, x, which may be a 
scalar or a uid. It creates a new copy of the record, R', with field /in this copy having value x 
We mention the replace instruction here mostly for compte^tiess as it will not be involved in 
the design of the backup and recovery algorithms we develop laten&£he thesis. For a detailed 
semantic description of this operation, the reader should see [17]. 

The select instruction is given a record structure and the offset in the record of the field 
to be selected. If the item to he^elected is an early completion structure, then the instruction 
queues itself onto the ec-queue. When the value of this field is finally known, the select 
instruction will be placed again on the enabled instruction queue so that it may execute! It is also 
possible that the kern being selected may be a suspehslbft \f M record fefiart of a stream. Recall 
that the rest operation on streams is translated Into a sctect operation 6t\ Pitie si^hd component 
of the head of the Current stream, Because streams are prttouceliln a demand driven manner, 
this field may be a suspension in which case the SELKrf'ftlsttticttoli 1 wittheed to send asighafto 
the instruction referenced by the suspension. Thef field occupied fey the suspension will then get 
changed to an early compiction queue which will get stT irr the activation responsible for 
priKlucing the next stream clement. 
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if /.opcode = select then 
let 
R - //(/.opt) 
/ = /.op2 %/must be a natural number 

' = *</) 

Newstate = if / € U„ A H(t) € ECQ 
thenCto, 

NewHeap(H, t, //(/) U {<«^, * F /}) 
NewEis, 
Env> 
ebeif/€U H A //(/)€ SUSP 
tbentet 

<«V*> = /f(/) 

SendToDest(Act, NewEis, u\ <uncon4, yt,'signal>, signal)) 
in 

NewHeap{H, t, MakeECQ (<u Fjt , k FA >)) 
NewEIS; 
Env> 
endlet 
eke let 

i4rt', NewEis' = SendVoJueiAct. NewEis, u FA , destinations, t) 
Act, NewEis" = SendSignaK Act', NewEis , u FA , destinations) 
in . — ■ 

<Act",ff, NewEis", Env> 
endlet 
endif 
in 

Afevwta/e 
endlet 
endif 



15.43 The Set Operation 

The majn operation on early completion elements, is the set instruction, The instruction 
takes in three arguments, a record R, an ofTset ia^ie recqrd wjuci^presents the field which is to 
be set, and a value, x. When the set instruction executes, it replaces the early completion 
structure found at the specified component with x Moreover, all the elements in the 
ec-structure are appended onto the enabled instruction tjueuesin^ the value of the field which 
these instructions initially requested is now available. The SET instruction, unlike the REPLACE 
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operation mentioned above, does not eause a new version of die structure to be created. Instead, 
the early completion structure is replaced with the value i& situ. His does not violate any 
principle of referential transparency because no instruction Is allowed to read a ftetd which is an 
ec-structure. Since the select instruction on structures prevents any of its targets from reading 
tiie value of the field until it is properly set, the applicative property of the base language is not 
compromised. Because all instructions which require the value of this field are on the early 
completion queue, SET does not send any results to any of its destinations, only signals. 



if /.opcode = set then 
let 
u = /.opt 
R = H(u) 
f= I.0P2 
X = /.Op3 
u'=H(R</» 

V v€N 

/?1(v)=/?(y)tfv*/ 

= x otherwise 

Act', NewEis' = SendSignal(Act, NewEis, u FA , destinations) 
H' = NewHeap(H, u, /T) : ^ 



<Act\ 
H\ 

NewEis' U HM 
Env> 

endlet 

— ji< 
emtti 



15.4.4 The Suspension Operator 

The SETSUSP operator i? sesponsiWe lor setting a suspentKM in a record structure. It takes 
in three arguments, areeordatrucjfcTre representing the head oflfce -current sffeam* an oflfcetiittto 
tiiis structure where the suspension is to be placed, and [ att*nstrtiatk»jaddress.i, representing; Ule 
instruction which is to be signalled when the suspension is accessed, Thejoffcet must fce an early 
completion element presumably constructed by a MAWa^fiehwtruetianr SBTSusPsel&in this 
record the value, <u f4 . i>. u rA being sthc u»d of the activa*«a«jtftWhioh the sersusi? operator is 
executing. If the ec-structure is not empty, then SEHiUSfrfignalssifiC activation as well since such 
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a situation implies that some select operation has already attempted to read the next stream 
element Like the SET instruction, SETSUSP does not send any results. The instruction is only 
used in the translation of the VimVal operator, affix, which is responsible for the construction of 
streams. 



if /.opcode = setsusp then 
let 

R = H{u) 

f= iopz 

i = /.op3 

Act', NewEis' - SendSigna^Act, NewEis, u FA , destinations) 

Vv€N, 

RXv) = R(v)ttv*f 

= AfakeSusp(<u FA , i>) otherwise 
in 

if*(/)€ECQA|*(/)| = 
then <Act\ 

NewHeap(H, u, R\ 
NewEis' 
Env> 
else let Act", NewEis'- 

SendToDest(Act', NewEis', u fA , <uncond, i, sionol>, signal) 
in 
<Act, H, NewEis", Env> 
endiet 
endif 
endiet 
endif 



2.5.4.5 Function Application and Return 

The instructions which will be of greatest interest to us in die coming chapters will be those 
concerned with the manipulation of functions. There aft fcUr instructions in Vim which deal 
with this: apply, tailapply, streamtail, and return. T*e Am.Y instruction is the standard 
function application instruction, taking a function closure and an argument record and 
constructing a new activation tor this function. By convention, the first operand of the first 
instruction in the activation receives the closure of the function, thus allowing the activation to 
access the free variables of the function, the first operand of die second instruction receives the 
argument record, and the first operand of the third instruction in the activation receives the 



§ 2 - 5 SEMANTIC FUNCTIONS FOR Ml 



37 



destination Ust of the apply operator, apply uses an auxiliary function, MakeDest which 
packages the destination entries found in the desthiatiofrlistof the instruction into a record and 
places this record on the heap. MakeDest takts in three arguments, the current heap, the uid of 
the current aetjyation and the destination component of Hie inttwcUon. It returns a record, a, of 
two elements, thVfim element contains the uid argument, and the second contains the tiid of the 
record containing the elements found in the destination list The uid of a is passed as an 
argument to the called activation. Placing the destination components into a record allows them 
to be accessed by the return instruction in the called activation. 



if /.opcode = apply then 
let 
C =/.opi 
arg = 7.op2 

<u f ,free> = H(Q 

u' = a new uid from IL , 

u* = a new uid from \j h 

Act' * AMAct(Act, u', H(u f )) 

IT = AddHeap(H, u", MakeDest(H, u FA , LdesOist)) 

Act", NewEis' = 
SendToDest 
(SendToDest 
(SendToDest 
(Act', NewEis, u\<uncond, 1, opl>, Q, 
u, <uripondL z opi>. fli^ 
u, Kuncond, 3, opt>, «") 

in 

<Act", H, NewEis, Env> 
endlet 
endif 



There is no explicit iteration construct in either VimYao or die base language. Instead, 
iteration is modeled using tail- recursive functions wherein the result of the iteration is obtained 
in the final recursive call. Otherwise, while the recursive call is being processed, the calling 
activation would exist merely to route the result back to the caller. To avoid having the calling 
activation persist until the call is Complete, there is a special ba^ language instruction for 
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handling tail-recursion, tailapply. The tailapply operator differs from the apply instruction 
in that it requires a third operand, which is the return address to which the result should be sent 
By providing its own return link to the callee, the caHer need not wait for the recursion to 
complete. We illustrate this process in Fig. 8. The tailapply instruction sends only signals to its 
targets. Typically, the target of tailapply will be a release instruction, described below, 
which reclaims the space used by dm activation. 



closure 



closure 






closure 










f 
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» • 1 










At* Record k 







LinkL 
Figure 8: Tail application in Vm 



First k-1 activations can be reclaimed without waiting for subsequent calls to complete. 



if /.opcode = tailapply then 
let 
C = /.opt, 
arg = /.op2 
dest = /.op3 

<u f free> = H(Q 

u = a new uid from U A , 
Act'= AddAct(Act,u', H(u f )) 

Act", NewE'a' = 
SendToDest 
(SendToDest 
(SernfroDest 
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(Act', NewEis. «V<unconiCl,opl> r jC> - 
u, <uncorut, 2, opl>, a/g) 
«', <uncorut, 3, opi>, </«/) 

i4c/'", NewEis" = SendSignaKAct", NewEis', u fA , destinations) 

in 

<Act"\ H, NewEis", Eni> 
endtet 
endlf 



39 



The streamtail instruction is similar to the f AILAPPLY operator in that both are used for 
implementing tail recursion. T\& ST|UZAMJAIL \ instruction, however* is sxs&A in the 
implementation of a stream producer. Unlike; tie TAiLAFfJLY operator, the return link 
argument to streamtail is a record fietd miheia&:strafeun*8lem^ When the next 

activation of the producer is instantiated, this return link wij get SET to the uid of the new stream 
element Thus, while the destination of the TAlp^^i^s^uc^on is always an instruction, the 
return link of the streamtail operator must be arxwd fi el dof the last stream element created. 
The basic structure of a stream producer using the STREffiAUulslruction is shown inlNg. 9. 

The return instruction takes as input two arg^e^^i^,Q^reUjr4 % ^4dresses and a 
value. It sends to each of these return address e s, it* specified value afHtoheiF sends signals to 
target instructions within its own activation. Unlike any other base language instruction, the 
execution of the RCTU^,ot^tor v? f%$^^ Thus, 

having the return operator execute may lead to instructions in other activations becoming 
enabled. The value argument to the return instruction repi^f^ ^^e^of the, activation; no 
other effects of the activation will be visible outside of the value sent by the rettt*rt ; 6perator to 
the receiving instructions in the calling activation. This property is a consequence of the 
applicative property of the base language. The RETURN instruction uses an auxiliary function, 
GetDest, which is the c«i*l«i«em of ifc* -^ when 

given the heap and ***id of thfe Record; fettrttts a sefi#«eii«Shg^ destinatidn Ifet packaged 
in that record. ..' ■-^.r.v^r 'H«.i>v^ >;i j: 



■ ■ - ' . ■-.■ ' " ' :i ' " ' . .." " . ' ' ."..' " I' " ■■' ■ i , i I.. , i .jiillli ii IJ li t i l . ii ] m i.ii M iii '. 

if opcode = return then 

DL = H{ /.opt) % the list of retont addresses 
w. = DL (i) % uid of the calling activation 
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Figure 9: Skeleton ofa Stream Producer 



targets = GeiDest(H, DI/7)) 

val^ljip2, %tlieva*ie to fce returned 

^c/', NewEis' = SendValue(Act, NewEis, u f targets, vat) 
Act", NewEis", » Sen&ignaRAcf;fihrt&,tifoiiestiMtions) 

in 

<Act",H, NewEis", Env> 
endtet 
endif 



The last instruction we shall present is the release instruction. Unlike any of the other 
instructions presented thus far, release is not used M-lhr implementation of any VimVal 
construct Instead, it is used for memory management purposes in the machine. When all 
instructions within an activation have completed, die storage occupied by the activation may be 
reclaimed by the system. The release instruction perfonhs this function. In a language 
without early completion structures, this operation would usually be a part of the RETURN 
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instruction, but because there may still be computations still in progress within the activation at 
the time the RETURN executes, it is necessary for a separate instruction to handle storage 
reclamation. 



if opcode = release then 
<RemoveAct(Act, u-J 

NewEis, 
Env> 
endif 



2.6 Summary 

In this chapter, we have presented a formal operational model for the Vim computer 
system. We introduced the application language, VimVal, and described the role of the Shell in 
the system. A rigorous definition describing the behaviour of some of the more interesting base 
language instructions was also presented. There are two key points raised in this chapter that 
should be noted. First, for the most part, an object created by an instruction is immutable. For 
those cases where it is not, as in early completion structures, the access and updating of these 
objects is carefully regulated to prevent incorrect information from being read. This feature of 
the system has major ramifications for the design of a backup system because it means that 
objects copied onto a backup storage device will, by and large, never need to be updated. The 
second characteristic of the system is the power of the individual base language instructions. 
Because of the expressive power of the base language, it should be possible to integrate the 
design of the backup and recovery system within the base language itself. The means by which 
this can be done is addressed in the following chapters. 

We are now ready to develop the backup and recovery algorithms for the system. In the 
next chapter, we give a general overview of the approach we take in designing these procedures 
and the enhancements which need to be made to the system in order to support them. In 
subsequent chapters, we shall use the formal model given here to precisely describe the 
algorithms as well as to show their correctness. We will formalize our notion of "correctness" 
later in the thesis. 
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Chapter Three 
The General Strategy 

The goal of this thesis is to design efficient algorithms whjch guarantee complete security 
of all information in the Vim system against toss or corruption because of hardware malfunction. 
In this chapter, we discuss the general approach Jr^t we sljall take in formulating these 
procedures. In the next section, we present a failure model of system operation. This model 
defines the appropriate context in whk& to reason about the deaigft of the backup and recovery 
algorithms. In section 3X we raise some funaamenua' femes ifcatrnust $k addressed by the 
backup and recovery system. These issues are concerrN* wfch when backup is formed; how 
backup procedures are invoked and how the transteof inibrmtton fam the mam memory to 
the backup store is handled by me backup svstem. We- shall be considering the probfemofdata 
security in the context o£ a single user system m whk* non^dwermhKrte computation is not 
allowed Section 3.3 gives a high levet design of uw bectotp= and recovery algorithms. We 
classify the information found in the Vim state into two different categories and discuss how the 
information in each of these categories is viewed by die backup procedures. We also describe 
the basic operation of the recovery algorithm in this section. Section 3.4 discusses the 
architectural enhancements that need to be made to the basic Vim architecture to efficiently 
support the implementation of these procedures. These enhancements are primarily concerned 
with the physical organization of the backup store. The last section isa.sujnniar^ .of the chapter. 

3.1 Failure Model 

Many of the decisions that are made in the design of the backup and recovery system 
follow from the failure model that is assumed. A failure model i?| a specification of hardware 
behaviour characterizing the type of faults expected and the interaction between failed and non- 
failed components in the machine. Some of the factors which will influence the design of the 
backup and recovery algorithms that are described by the fal lure nrodel include the frequency of 
lailures in the system and the level of hardware error dctodion capability that isprovided. 

Vim is not a fault*tolerant system and, therefore, there will be faults that are not masked 
which will cause the system to behave erroneously. It is unreasonable to expect, however, that 
there will be no fault coverage in the system at all: like many conventional systems, Vim is 
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expected to provide enough fault coverage to correct many common errors arising from minor 
transient faults. Correction of single bit errors in memory, for example, is a feature which is 
found in many commercially available memory, units anil, thus, the services of the recovery 
utility should not be required when such an error is detected. In this thesis, we shall assume that 
the recovery utility is invoked only when errors cause information found on main memory of 
secondary store to be lost or corrupted. Power outage, short circuits, a malfunctioning disk head 
etc. are some examples of the type of faults which lead to such errors. 

We do not expect that such faults will occur frequently; hardware is assumed to be reliable 
most of the time. We do make the assumption; however, that invalid information created 
because of an error is detected when it is accessed. For example; ifa faulty disk head causes data 
to be written incorrectly onto disk, men when the data is read at' some future time, the error will 
be detected. This assumption is important because it means that any information which the 
backup system observes and copies will either be corrector detected as being erroneous — 
invalid data is never maintained by the backup utility. 

If the recovery utility is invoked, it will need to reconstruct the system state based on the 
information preserved by the backup facility. During this period, another failure may occur; the 
recovery facility must be robust enough to correctly restore the system state even following such 
circumstances. We discuss how this may be achieved later in the thesis. 

3.2 Fundamental Issues 

The backup system will need to interface with the interpreter and shell to monitor the 
progress of computations in the system. We need to decide, however, when it should actually get 
invoked and by whom. Secondly, once it is invoked, what information should it actually copy to 
the backup store? Thirdly, how should this copy operation be performed Le. could normal 
system operation be intermixed with the execution of the backup procedures or must normal 
processing cease while the transfer of data is taking place? 

The first question was already partially answered in die previous chapter where it was 
mentioned that the semantics of die base language instructions could be suitably altered to 
support the backup procedures. Unlike most conventional systems, the backup utility is not 
explicitly invoked by any process or user; it is implicitly activated whenever the appropriate base 
language instruction is fired. In a sense, the "logic" of the backup algorithms is distributed 
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among the various base language instructfon^ of whkh process 

activates the backup facility is not germane under ihjg design. Np, one process is responsible for 
invoking the entire utility; different portions of JfoMkiQ procedures are activated as 
instructions in an activation are enabled, to ti^ sense, Jhe backup |acjKty Js m0 re an extension 
of the interpreter and shell, rather than as a separate pn*r»m which ^periodically invoked 
according to some predestined policy. Vim offers teemmte m make the backiipfedlity 
more efficient because it is embedded within the, mterprct* «d ahett This orgaiiizatioa will 
allow us to design a backup facility that can oj^eihej,*^ of opm^tation te a greater 
degree than would otherwise be possible. 

The answer to the sectfhd question involves (fete^ihinf ho* much information should be 
preserved and how the remainder of tfie sysieih 'Itytftfj "Be derived from this data While all 
data generated by the system could conceivably be copied onto the backup storage > medium, it 
wouid not be a very practicaLsplutioii because of the ^tf^^cug^^^^^^ instructions 
in the base language generate data and up$*e,*h» » a^^Jeffie^ation of,*i»««tegy 
would involve modifying ey*ry base language iaate^fa mtem**to&*#. execution to 
backup store. This would clearly result in severe performance degradation. The backup 
algorithms will, therefore, need to mairuain infonnation about, the system state in* condensed 
form which the recovery system can subsequen^ ; W M ^ve||h^ pan of the system state not 
explicitly preserved. As we shall see, most of uw.design effo* for <he backup, and recovery 
system will be devoted to devising, efficient algorithms to maw*in an4 ; intejpret these records. 
We discuss this issue in greater detail in the next seotioa. 

Because Vim is an applicative system, no data found On the heap, once created can be 
subsequently altered by either the backup system of th£ mfcrprerer. Thus, having the backup 
procedures operate concurrently with normal sysrem Op^ration^rtoVcau^ any invariants over 
the data to be violated. Moreover, the data coptetf *by the ^m^tem willhever be in an 
inconsistent state when me copy operation is iJerfofn^'teclui^hd updating of information 
takes place. Our backnp procedures canV therefbre; b^altowid '& execute concurrently with 
normal system operation 1 without the need for any explicit ootislfl&ey fchecks. 

During the recovery process, no shell commands are^acccpted by the system. If shell 



There is a caveat to this claim which will be explained in the next chapter 
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commands could be processed concurrently with the recovery process, it may be possible to have 
computations reference data which has still not been restored by the recovery procedures, in 
addition, this restriction also simplifies the interface between the recovery procedures and the 
shell by avoiding the need for any synchronization protocols between the two processes in 
updating (or deleting) environment entries. When the recovery procedures are invoked, they 
make no assumption about die integrity of the data which may stilt be accessible. Thus, the only 
information used in the reconstruction of the state is that found oh the backup store. Of course, 
it is inefficient to restore die entire state of the system if only a fraction of it were affected by an 
error. Significant complexity is added to the backup and recovery procedures, however, if we 
require the system to support partial recovery. It is not clear whether the benefits derived from 
implementing partial recovery outweighs this increased complexity. We shall address this topic 
again later in the thesis. 

In the next section, we present the high level design of the backup and recovery 
algorithms. The rationale lor our design decisions have been mainly based on the effectiveness 
of these algorithms in addressing the questions raised in this section; • , ■: r 

33 A High Level Overview of the Backup and Recovery Facilities 

The design of the backup and recovery faculties are based oh one important observation: 
every computation in the system is associated with the evaluation of some shell command input to 
the system. Thus, one immediate solution which presents itself is to simply record all shell 
commands on the backup state. This is obviously a correct solution since the behaviour of the 
system is presumed to be determinate. Reexecuting, in the proper serial order, the shell 
commands that were input to the system before the failure occured is, therefore, guaranteed to 
yield a correct state. This state will be identical to the state immediately prior to the failure 
except for the uid's associated with structures and activations. The uid's chosen during the 
recovery process may not be the same as chosen originally 2 but because these uid's are not visible 
to the programmer, no difference in the two states will be externally discernable. Obviously, 
such a scheme would inflict little degradation to system performance since only the text of the 
shell command need be maintained. On the other hand, recovery would be intolerably slow 
because every shell command is reexecuted from scratch with no information about the results of 



Recall from Chapter iwo lhal no restrictions are made on how aid's may be selected 
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these commands being kept on backup store. The recovery system, starting from some initial 
recovery state, would need to reesecute every shell oommand iup«l to the system from the start 
of system operation because no information about the wsttK of any of these commands are 
recorded. Such a major drawback makes this strategy unattractive for all practical purposes. 

To see what optimizations can be made to, alleviate this problem, let us examine how shell 
commands are used to alter the system state. The shell cotimaod of interest tous here is the 
bind command. The BJNDcommand biods a name to the result of some compulation and places 
this binding in the user's environment. The value bound to the name represeots a kmgihved 
value — it survives the computation in which it was coated The data seen by the user of the 
system are precisely the values iboujndm his en^ 

greater than, the compu|»tio»s m which they we* defmed. it would certainly be a major 
optimization if the backup facility mawtaiaed these ttafejen on the backup atore. This would 
obviate the need for .the iesov*ry system to iee«cute those commends whose evaluation 
produced these values. TAe backup facility must now, in addition to mamtaining a tog of the 
commands input to . ; tfae system, abe reconi d&<mma,*tt*> bindings found in the user 
environments. These bindings cannot be arbirtarily changed because $m u applicative; thus, 
once a binding is recordedon backup store, the bacte^ewlity need never again reexamine that 
entry in the user environment to check if it has been altered. 

The datt found in Vwnuy be classified mto two ^ 
to the result values of computation associated wm i^ bound to a 

name in a user envircmmcitt» pd^ that are either 

part of sortKi active comjwtatt^ yet been bound 

to a name in some environment A computation consists of the collection of activations and data 
created during the evaluation of a shell command. Instructions in tiiese activations produce 
transitional data since mis (lata will survive for only so long i as uW computation in which it was 
crMed exists! The Value ffiuuly produced by a computation will get bound in an environment 
and, as a result, wilt become quiescent The oacltup falfftty. according to the scheme presented 
above; would only be aware of quiescent data. Transitional information corresponding to data 
produced during a comfetau^ would hot be : uno^rlne scrutiny of u^ backup procedures. 
When a failure takes place and the system state needs to be properly restored, the recovery 
facility first restores all qutesceht^data preserved by the backup facility onto the new recovery 
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state. It must then reexecttte those shelleommands found on the^mmand fog which hadeither 
not yet competed or whose result binding coutd not b« recorded by the backup facility before 
the failure. These shell commands will be referred to as >ofa»7g co i tt m ands . 

Having the backup facility record only quiescent data is an optimization that reduces 
overall recovery tune. It does so without excessive peffertwtecc degradation because the 
backup facility is only invoked when an WVt®fa*rWtoi&cM 5 e%eeute^ ttf actuafly place the 
binding in the environment and, moreover, can perform the copy of me data in parallef with 
normal system operation, ft would be an even gfeatW^ep^Mfa^wlf the backup svste 
help BJdaee reexecution time erf tiwse voladle ^dl c unwn a nds found on the tog by recording 
information about those computations which were active at the4!Bhe%f fainSfe. if there are many 
resource intensive, time-consuming activations*! #eo^pm^al^ j iie^ f havii^^hebackm5 facility 
ignore die presence of the transitional data pro(h*ced in thi^computadon ^neans that the time to 
lecover this state must be at least bounded frol^ljeftrtf'fty^lte time it takes to reexecute this 
entire computation. This is not very desirable since u^e-«omp«1auon could have ahtady %een 
executing for a very long petted when the^sfaSiii!*^ 

computation on the backup store would altow the 'fgftoWjf j Hiefl$r to arbftf heedlessly 
reexecuting those subparts of a cwnputation which had ah^y produced their resuft beforfe the 
failure. 

it is reasonable to expect that there will be many computations in progress at the time of 
failure. Toteco¥dthepiOg#**aofthes^^ 

about these computattons^acwnpiift^ structure of a 

computation lecofdanaftoweaaefciati^^^ the next 

chapter. -' v ' *n •^'■''■ V;:r 

We can now present our intended model of system operation for Vim. As computations 
complete, causing values to be bound within stmu? CTyjr^incn^the.b^l^ip facUity preserves 
these bindings on the backup storage mc4ium. In ^dditton, ,%|re w^ajso fcjtyany^ve 
computations in prepress. The backup faciUty maintains uifoEmati^^ab^t^ 
as well. This information, embodied as a computation record, can be used by the recovery 
procedures to avoid needless recxecution of computations yvh4c^, h^dl > ^t|'e^td|^ T jp^duopd^^ tf^eir 
result before the failure. When the recovery system is iny^ced, it fir§t restorer al| quiescent data 
found on the backup store. It then uses the computation records to restore the remaining part of 
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the state. The state after recovery is complete will be equivalent to the state which existed prior 
to the failure insofar as the structure and information content of both states will be the same. 
The states need not be identical; however, because me uidV lislobiated wiUi activations and 
structures may be different. The reason why the states would ntit be identical is because the 
order in which enabled instructions are cftc^ -for execution may be different during the 
recovery process than before the failure. This does hot cOThjJrOmise the correctness of the 
recovered state because of tfie applicative nature of Vim ^ no side-effects occur and, thus, no 
explicit ordering on instruction execution needs to be adhered ! tb. We illustrate the system 
operation in Fig. 10. 
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Figure 10: System Operation 



3.4 Architectural Enhancements 

Up to this point, we have only mentioned that the backup store on which backup data is 
kept has the property that information entrusted to it will survive failures of the machine with 
very high probability. The most common type of backup store used is magnetic tape. The 
sequential access nature of tape drives, however, makes i it mcbnvienent to update the 
information found on the tape. Since the backup facility will be frequently updating 
computation records associated with active computations to reflect the progress of the 
computation, using tape as the only backup storage device would "be impractical. Our design 
dictates the need for a fail-safe storage device from which information may be easily accessed. 
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updated, and deleted. We call a device which has these properties a stable storage device. It is 
not difficult to implement such a device on top of non-st#ble storage devices. Lampson 
{21] gives one implementation of a stable storage device, in wW$h disk storage is converted in to 
stable storage by maintaining multiple copies of the clata$n discern disks and ensuring that all 
writes to disk are atomic Le. the write either takes plaqe on both disks or on none. Because both 
disks are guaranteed to have consistent infoanation* datafkwtbeauise of failure of any one disk 
can be recovered from the other. Advances in Vl^I teciujology have alsoinade it conceivable to 
consider a hardware implementation of stable stora$||tsinjfe fojjexampJe, CMOS static RAMS 
and a backup battery supply. Because of the low power consumption of CMOS chips, 
information on RAM could be retained, despite power failure, using the backup battery supply. 
In this thesis, we shall not be considering implementations of stable storage but will assume that 
such a device is available for use by the backup and recovery utilities. Because of its relatively 
high cost, we shall also assume that stable storage is not very large (certainly much smaller than 
the size of the backup state) and, therefore, in order to guarantee that backup information is not 
susceptible to loss, it wHlbe necessary to have another backup storage device capable of holding 
that part of the backup state which cannot be hejd instable storage. We assume magnetic tape 
storage is used for this purpose. 

Quiescent data is never updated by the backup procedures and, therefore, can be kept on 
tape. Of course, if the binding is subsequently deleted, the data will have to be removed from 
the backup state as well. A delete record indicating that a value has been removed from the user 
environment can be written onto tape in such situations. The computation records associated 
with active computations do need to be accessed and constructed relatively frequently. These 
records will, therefore, need to be held on stable storage. In addition, the command log. which 
contains all shell commands input to the system whose results have either not yet been produced 
or have not yet been recorded onto backup store, will abo need to f tie held on stable storage. As 
we shall see, most computation records will be relatively short-lived and, thus, will not occupy 
stable storage for any significant amount of time. Tjhe final value of a computation record is 
quiescent and can be migrated onto tape, allowing the space used by the computation record to 
be reclaimed. It is expected that stable storage will always be able to support all compulation 
records in the system because of their short lifetime. When the recovery system is invoked after 
a failure is detected, it will first read from the tape all the quiescent data and will restore as much 
of the environment image as possible from this data. Volatile shell commands arc then executed 
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Figurt 11: High Level Organization of the Backup Store 



Transitional Data 



from the command log in the order in which they were originally input The computation 
records found on stable store are used to reduce the overall reexecution time during this phase. 
When this phase is complete, the system can proceed with normal. operation. The high level 
organization of the backup store is depicted in Fig. 11. 

We illustrate the organization of the Vim system with the backup heap and environment in 
Fig. 12. The backup heap is used to hold all transitional data whereas quiescent data is held on 
the backup environment The backup heap and environment constitute the ViM backup store. 
The Interpreter constructs computation recordsman the backup heap during) normal processing 
and interprets them during recovery. In addition, the imermedi«c results of a computation are 
also stored on the backup heap by the interpreter, <A^&*^ owdjngsarc placed on the 
backup environment by the Shell which also builds the command log foundon the backup heap. 
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Figure 12: Abstract Architecture of the Vim System with Backup Store 



3.5 Summary 

In this chapter, we have presented a high lever strategy for the backup and recovery 
algorithms for the Vim system. Ottr main observation about system behaviour was that all active 
computation in the system is associated with the evaluation ofsome shelf command. The first 
solution proposed involved simply storing the log of all sheli commands, reexecutrng them from 
the beginning if a failure occurcd. While correct, because Vim is a determinate system, this 
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solution has the drawback of a very slow recovery time. A major optimization to this solution is 
to record all quiescent data Le. data bound in some user environment A further optimization, 
intended to reduce the overall recovery time in reexecuting volatile shell commands is to have 
the backup facility maintain some measure of information about all active computations. The 
recovery facility uses this information to avoid needless recomputation. Once this reexecution 
phase is complete, the state of the system is properly restored. During this reexecution phase, 
the order in which instructions are executed may be different from the original execution 
sequence. This may lead to different uid's being assigned to different structures but the overall 
structure of the heap and activations component remain identical. The reason why the order of 
instruction execution is not important during the recovery process is because Vim is an 
applicative system. 

We also introduced the notion of stable storage in this chapter. Information about active 
computations will need to be frequently recorded by the backup system. A backup storage 
device on which data can be easily accessed and added is required to support these computation 
records. While quiescent data can be copied onto tape storage, computation records need to be 
maintained on stable store. 

In the next chapter, we present the detailed organization of a computation record and 
discuss how the backup system monitors the progress of computation. As we noted earlier in this 
chapter, the logic of the backup procedures is actually distributed among certain base language 
instructions. We present a formal model of the Vim system supporting the backup and recovery 
procedures and argue that the information embodied in the computation records is consistent 
with the actual system state being represented. 
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Chapter Four 
Constructing Computation Records 

A major aspect of the backup and recovery algorithms for Vine concerns the construction 
of computation records. Recall feom the last chapter that a oomputation record is used to record 
information about currently executing cotapulatiDas. ? to this chapter, we shall be primarily 
interested in how compulation records may he constructed and maintained. In Section 4.1, we 
present the ab^ractrepreaemation of computation records. The main component in the record 
is known as an activation descriptor entry which embodies staU inftuimaUon about an individual 
activation, In order to construct a computation record, changes to the operational behaviour of 
the base language instructions given in Chapter Tw© will be necessary. Section 4.2 discusses 
these changes as well as change* necessary to the shell and interpreter. In section 43, wefprtsent 
me altered operation of the base language instructk)ram teims of an abstract operaUonal models 
MR, which is an extension of model Ml presented in Chapter ttvo. Them are several major 
optimizations which can be made in managing computetion records. These optimizations are 
also formalized in this section. 

4.1 The Computation Record 

In Chapter three, we argued that the backup system should record the progress of active 
computation in the system to help reduce recovery time. A computation record is an 
information structure constructed by the backup system for this purpose. Coir focus in this 
section will be on determining how much information should be kept on the computation record 
to allow the recovery procedures to restore the system to its state prior to the failure. One simple 
scheme would be to periodically .checkpoint all activations created by a computation. To 
checkpoint an activation means recording the state of all instructions which have not yet 
executed in that activation at the time of the checkpoint The state of an instruction consists of 
its opcode, destination list operand and signal count, as well as the value of its operands*. This 
approach would be very similar to that taken in many other parallel computer systems where a 
recovery point representing the state of one of possibly many concurrently executing processes is 



If an operand is a complex struct we. Uife mains recording M substructures referenced from the -top-tev«l structure 
as well. 
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periodically taken by the systems'; backa> Jacilil|y lln* our system, the recovery procedures would 
only need to find all enabled instructions in the computation records to begin the reexecution 
phase. :V ■ j A'-.i^:'l^uy .- 

Periodically recording the state of all activation* created by a«on*petation is simple idea 
but has two major drawback* which does no*jna*te4ia tm&fa^utte*mcfapiirp6isiL First, 
checkpointing all acuvatk>nsin aojniputation will probaMy^e doo«ry task because die size of 
activations can be very bigi Secoodiy, in cedes :«HpfmMMfJttie Scot^ste^t inline of the 
computation is maintained on the backup utility, we would have to dlsaflow ahy- enabled 
instruction within any activation in th^computatkHf ft^>e>^tmg^vh^^^ 
activation is being performed To see why this is the case, consider twtf activations, d arid p in 
the same computation whete e*hsftctM7fc IC^is allowedto<§**nHB while aeheckpomtof a is 
bemgmftde» then fl]ttm$atw'to*tol^&^t1mmm&wk K£fc£As£ operation; If the 
image of a on backup state does not reflect th«>*ewtt 

checkpoiated before the release operation executed, the computation record would represent 
an incorrect state. Upon recovery* there would be n© way ^mtmtt^tt^m^^hy^Mbffi 
without reexecuting a. This is precisely a manifestation of the pn&Mems encountered in other 
concurrent systems that use recovery points to guarantee data security. 

A more clever approach to recording state information about activations takes advantage of 
the applicative programming model Vim uses. A distinctive feature of an applicative language is 
that each function can be treated as a constant applicative form caf. A caf consists of constants 
combined by function composition and application. In conventional programming languages 
such as Pascal or Fortran, a function cannot be treated as a constant because its evaluation may 

- • :• , , ..■■■:'...).,■■ ■ --; AY- bvM'.f- ■ •'- ('rO^Vv:-""! '■• - "- ..'■'>,■•■■:/>■ • 

cause side-effects to occur in the program. In Vim, having alj functions be simply constants 
means that the behaviour of the function can be determined by just knowing its inputs. In the 
base language, an activation is the application of a function to some input Because functions are 
cafs, we can embody an activation on the backup state by recording the function closure, its 
inputs and return link. Under this scheme, the recovery system would need to only apply the 
closure to the argument list to construct the corresponding activation being represented. An 
important advantage of this proposal over the checkpointing one is that no executing code is 
maintained on backup store. Because all data is immutable, the transfer operation of the data 
from mam memory to backup store can proceed in parallel with the execution of jury activations 
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which operate on this data Moreover, the amount of information which needs to be copied is 
also greatly reduced since no data created by instructions within these instructions are preserved. 
Such data would be recovered when the activation is reexecuted. 

A natural representation for a computation record in this scheme is as a directed tree in 
which nodes represent activations and edges indicate caller/callee relationships between pairs of 
activations. This tree is known as the computation tree for the computation. We illustrate this 
representation in Fig. 13. 
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Figure 13: Representation of a Computation Record 
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The root of a computation record represents the initial activation constructed by the Translate 
function of the shell Every node in the computation tree is labeled with the uid of its 
corresponding activation. If («/) is a member of E c , the set of edges in a computation tree c, then 
activation / is instantiated from activation s. Each computation has a unique computation tree. 

A node in a computation tree is called an activation descriptor entry. An activation 
descriptor contains the necessary information about an activation needed to restore the state of 
the activation. The representation of a computation record given above is simple but certainly 
does not help much to alleviate recovery lime for transitional tfcrta. This is because -computation 
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trees may become very large for long running computations. Reducing the size of the 
computation tree would speed up the recovery process. The information in an activation 
descriptor entry in this scheme contains the closure and argument list of that activation. During 
recovery, however, every activation represented by an activation descriptor entry in the backup 
state would be reexecuted. The time to reexeeute all these activation? would result in an 
unacceptably high recovery time. Clearly, what is needed is a mechanism to record results of 
activations as well as their instantiations. Thus, when a result of an activation is known, it 
replaces that activation in the computation tree. When this value is encountered by the recovery 
procedures, it is sent directly to the destination addresses, eliminating the need to reexeeute any 
activation in the subtree rooted at the node containing the result In this way, we can imagine 
the computation tree growing and shrinking in response to the instantiation and completion of 
function activations. This process is shown in Fig. 14. 




Figure 14: Dynamics of a Compulation Tree 
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When an activation is instantiated, a new descriptor entry is placed in the computation tree 
and an edge is added from the caller to this new e^p. W4ierijth| insult of an activation becomes 
known, it replaces this node in the computation tree. .^ s«j>o$jn{|te activation descriptor 
entries are then removed. There are two features p|^ 

data backup strategies found in conventional Sfsjsera* The, Jest is th& thje updating coftfate 
information in our scheme is dependent totally, qn rp^ behayiQur. ^we had mentioned in 
the last chapter, typical backup strategies use %$i^e}#$^pp)iy tQ detemtine yivta the 
checkpointing's to be done, The second, an^^mp^iojpor^ the 

construction of the computation tree takes advantage of the applicative programming model, the 
computation tree can^e ^nmetTwheneve^resu^ In a 

language whfch permits sfcfe-eil^indii^ngW wl^ would hot be able to 

manage &e bSckup image of Ae^nipiMion^rHinls rrilmter. 



-K-t> 



It now remains to shovv ejcacfly how the^fe^prej^; s^and .base- language semantics 
need to be modified to sup^ Wi? examine the 

structure of an activation^ descriptor entry in gres^de$l j^th*n^secti£n. ; 

4.2 The Activation Descriptor Entry 

We had mentioned in the last section that an ^activatioii^des^r^^hould contain enough 
information so that the recovery procedures could, jsstose ti^s$te- ; ,ol the activation ^We 
observed that if the fraction closure ajuiLajprniem; ^^die^vati^ei* j>r^^ 
recovery procedures need only apply the dosurp to the argunieatlJU, setting til* jetum link, to 
restore the activation stafc. 

The recovery process m this proposal is^raightfbtward. A1f icttvatibn descriptor entries 
containing the function closure, argument record an^'retuhi' ttnk can Hfive the function 
application take place in parallel. Whenever an apply operation ft to be executed during the 
recovery phase, a check is made to see rf the ftirtctibri riBs alrlfia/be^ri instantiated; That is, all 
apply instructions during the recovery phase, ,^ec^, to sgejf an lactijvajiqo, descriptor already 
exists for the activation they are to initiate. If onf : exists we cfipief^ctiv^ly ignore the instruction 
since the result of the application is already known. If np sucJjdescrij^tor wistsv we perform the 
application. Result values found on an activation descriptor entry are used to prevent initiation 
or an activation. When a value is found in an ADE, it can be pent directly to the, destination 
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address specified in its return link 4 . ~ 

In a system in which errors requiring intervention of recovery procedures are assumed to 
be relatively infrequent, we may find the cost of even maintaining function closures and 
argument records too expensive. As vre explain beldw. the rtidih reason for needing the closure 
of the activation on the backup store fc if we wisli 1& have all activation descriptor entries 
evaluated m parallel. If we are willing to tolerate longer recovery time, we can signficantly 
reduce the amount of ^formation which needs' Id be rield on th^ activation entries by 
eliminating the need to hold even the function closure of argumeht record on backup store. 

Instead of having all activations initiated ,|n pafaUel, we cai^have the computation 
reexecuted from the initial activation descriptor m the^pujatioa tree. Whenever a new 
activation is about to be initiated. $e recovery pjpxsthjre fir% examines the oarisssponding 
activation descriptor entry for that activation (remember that there is a unique computation tree 
for every computation). If that entry contains a re^lttten Aai value !s used directly and the 
new activation is not initiated. If, on theo&ef ftandtn& result v^i^ls found in the descriptor, a 
new activation is constructed and processing proceeds as normal. No function closure or 
argument record needs to be maintained in this scheme because the computation is reevaluated 
from its initial activation — no parallel invocation of activations within the computation takes 
place. While the time to restore a computation fc greater than if function closures were 
maintained in the backup state, it Abounded bf thetirne the systeWwould have taken to have 
processed this computation under normal ^ drcumstatfcet if ronctM closures and argument list 
of activations were maintained, then the recovery system cOuki exploit wore parallelism man 
what was available during the original evaluation of the computation precisely because all 
activation descriptor entires in the computation record coujd, be evaluated concurrently. It is 
important to keep in mind, however, that e^eflif :; ^ : e^Jf^bfp$g4 : 'is not kept on the 
activation descriptors, the reexecution of me computotk^, would sjtill exhibit as much 
concurrency as it would have under normal conditions. 

The information held by an activation descriptor entry must aHow the recovery procedures 
to determine if an activation needs to be initiated or not There are two forms of an activation 
descriptor. The first form is for those activations whose results were recorded by the backup 
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We arc assuming (hat uid's of activations are preserved in die backup suite and are used during die recovery phase 
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facility. If an ADEfoT an activation a contains a valti^ thettihf^valiie reflre&htsthe result of a. 
The second fewn tifmABEte used wnen the res^fJ^Ftft dfMatiMaS not yet been recorded 
by tfie baoteip ftcilHy. In this case, the >!» ^MiiSiliaB 'IHi^'felifes^fiJi' aai'^dBh^&ibhs' fhaf'-Were 
initialed from « befert the faWrWoeefcrt^^Sffinf^^ 
edges in the computation tree refcr^tc^fa/cfHe^T^^ 
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uid 
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edits to dacerulott computation records. 
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tftfurv 75; Structure of an Activation Descriptor Entry 
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We represent an edge hj actiyatipn ^ descriptor entrjr ^^j^^^^^^^^yjtl^^set 
represents the offset in the activation array, of a.fanctionM»^ 
instantiates the new activation and where uid i^jttesents 1^ J^^f^, aj^y^^ i1 de 5 P!ll? tor 
associated with the called activ|tton. Jhe ? ^ u corn^^| u^^yjjen^ h the aeration 
being initiated. When an application instruction is encountered during recovery, the activation 
idesc^pir^cctfrespondiiig to^**v*ten to tfeffe&iaMif^^rrM 'Wtivifte is present in 
thfc*4Z», itcan be sentdirtctlf wOwdesdm^S*^^ is 

used to locate Ae*4J»obfn» attivatk* «^M EM g^^n<^'|^$ii'^Hy^ we need'tit* c&set 
£&d at ail is because theoro^ln whkfc*n«n^^ 
same as the order *» which they were ortg4rMdly- **ec*r**^ 

function application is ordered with respect to every ^eraft tt k iatiort ReCTeortibn of a given 
actheationiwifl not change this ordering. In a coricurfen^s^em sutfr a* Vr^'tMere floeSlfdt exist 
any a priori ordering on in*ru«ton execution, ^mm ffie^m^c^ rtuthbir, q^/.tjf the 
furaaio«fappikaaio»«p^ 
heap to pro^y-identi^ Ottering the reebve¥y pi»i^l< > ^ ^^'-^^- * ° f 

The scheme we have given above requires little intervention by the backup facility. 
Function application is noted by adding^ . new v .^ ! v #' 9^^f^P!lp to Jf JP ^ e WI^RP 1 ^ 
computation tree. When die result of an actiyatfon is known, H, replays the despr^ptpr erjuy fpr 
that activation on the backup state. We do not maintain argument, records and closures pf 
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activations, choosing instead to reexecute the computation from the beginning, only avoiding 
reexecution of activations whose results are already knpwQ. White recovery time in this proposal 
is greater than the one in which closures and argun^t iaoprds are maintained, there is a 
substantial reduction in the computational resources required by the backup facility. As we have 
mentioned previously, efTidemimplementipn. of this strategy will require alteration of some of 
the base language instructions. A formal definition of these operators is given in Section 4.5. 

4.3 Early Completion Structures 

The preceeding sections have presented &e general framework and rationale upon which 
computation records can be organized. While sufficient for; most cases, there are certain 
program structures for which our design is still inadequate. IfceHirst type of program structure 
not properly addressed in our presentation is ^he Sri structure. By itself, an early 

completion structure does not add useful information to the backup state since it only indicates 
mat an activation lias been initiated to produce the desired value Copying an early completion 
structure onto the backup store would not allow the recovery system to restore the proper state 
unless me activation responsible for producinj [the value which is to replace that structure is also 
copied. This is obviously not a desirable situation to have to deal with. 

For our purposes,, a sknpter {and more efficient) sokmion m to avoid copying early 
completion elements until all the early, completion fields itt toe smicture become set When a 
structure which cprtfainsearly cwnphrtjon gejdtisto be copied onto backup store, these e^fields 
are labeled with a special flag indicating thai me <»©^^ copied. 

When a set instruction encounten; such a field, it chwks to see whether any other early 
completion elements exist in the structure; if so, noeopy operau^; is performed; otherwise, the 
structure is copied onto the bacjtup.store, The ^£ wrtkh is to refererKx this structure will 
initially have its value field reference astructureMth^srjeeMivahie, notcopied. When all fields 
in the structure are known, this reference gets replaced with the<»efereHce to the fiiliy defined 
structure. If the recovery routines encounter a su-ucture with a value «OtCOpiett, it is treated as 
not being defined and is ignored during the reexecution process. The structure containing the 
early completion clement being set may be a component of a larger structure which also needs to 
be copied onto the backup store. If this structure becomes fully defined as a result of executing 
this instruction, then it too will get copied onto the backup store. We illustrate the effect of the 
SfT operator on the backup store in Fig. 16. 
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5 is fully dtfmcd by SET operations and is trwsfemd to backup store. 
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crripty 




u' 




Figure 16: The Effect of the set Operator on Backup Store 



4.4 Further Enhancements 

There are two other program structures whose behaviour cannot be efficiently captured by 
the backup facility by just modifying the semantics of the apply and return operators The 
first is the tail recursive program expressed using AeTAHi«WY operator. The second class of 
programs not handled by our system arc those involving the production of sireatns implemented 
using the setsusp and STREAifrAiL mstiucftofis. Both thm <tlffises of programs use special 
function application and signalling operations which t^uire m^^iataed algorithms than 
those presented above, la the next two^subaeotions we discuss how th* backup system should he 
augmented to handle tail recursive activations and demand driven evaluation of stream 
structures. 
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4.4.1 Tail Recursion 

Recall that tail recursion is used to implement iteration in the base language. The key 
feature of tail recursive activations is that they need hot persist until the recursive call completes 
because the return link is provided as the third operand to the tailafply instruction. In our 
current design of the backup system, if the tailapply operator was treated as being identical to 
the apply instruction, then every tail-recursive activation would cause a new activation 
descriptor entry to be constructed, The structure of the associated computation tree would 
contain a long chain of ADFs with only the last ADE in the! chain having the relevant result 
value. The backup system can optimize the construction of ApFs when tail recursion is 
involved by reusing the same ADE for tail recursive calls instead of building new ones for each 
new tail recursive application. An important observation concerning tail recursion is that tail 
recursive activations differ from each other only in then; argument records. All activations 
initiated from a tail recursive call use the same |uj$ti^$3sure and return link. Each activation 
serves only to construct a new argument record for the succeeding one to use. In fact, because 
activations in which the tailapply operator executes tto not return a result value, there is no 
return instruction which is executed. It should be clear that this behaviour is not well 
supported by our backup algorithms which very much depend on results of activations being 
recorded on backup store in order to help reduce recovery timei^vplatite commands* The 
reason for this incompatability is the fact that no tail-recursive activation except the last returns a 
result, making any intermediate tail recursive activation descriptor entries essentially useless. 

We introduce a new type of activation descriptor for* tail recursive activations which 
includes the argument record of the activation. When a function is instantiated by an apply 
instruction* an activation descriptor is constructed for k with type appty. tf this function was tail 
recursive, then during the evaluation of the function -a : TAHiAfffLY Instruction may execute. 
Execution of this instruction, white causing a new activation to be added to the set of activations 
in the system, does not necessitate a new activation descriptor to be constructed as well. Instead, 
we change the activation descriptor of the current activation to type tailapply. The argument 
record passed as the second operand to the tailapply instruction is recorded in this activation 
descriptor. In addition, all edges emanating from this ADE are removed. The old state of the 
activation descriptor is thus replaced to reflect the new activation. Other function applications 
that take place in the activation are recorded in the tailapply ADE as was done in the apply 
ADE. Subsequent tail recursive calls in this activation will cause the same effect as took place 
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initially:. the old argumeat record is replaced with the argument record of the new activation, and 
the edges emanating from the ADE are removed. 

The inclusion of the argument record in the descriptor allows the recovery system to avoid 
reexecution of all the tail recursive calls leading up to tite one represented on backup store. 
Since the closure and return link are the same, keeping the argurpf««f«cord in the ADE makes it 
unnecessary to reexecute any of the prior tail recursive activations originally executed from the 
initial apply. The representation of tail recursive activations we have chosen has two beneficial 
aspects. First, the depth of the computation tree is noUfterfased for every tail recursive call since 
the new ADE can replace the ADE of the calling actiy^oi ;i This is because tail recursive 
activations send their result directly to the address specified hi {heir third operand; the calling 
activation does not receive the result of the callee. Secondly, by storing the argument record on 
backup store, reexecution can begin by applying the function to this argument record and the 
return link provided by the apply instruction which initially instantiated this function. 

In Fig. 17, we show some steps in the transformation of a computation tree which 
embodies the evaluation of the following function to illustrate the process described above: 

Function Example (f: Function, n : Integer returns Integer) 

Function Taitexample(m,n : Integer, ^Function returns Integer) 
ifm>/i 

tbeom 

eke Tailexample(/(m), n) 
endfun 

Taitexample(l,nJ) 
enftfm 

4.4.2 Stream Structures 

Our basic approach to recording the progress of computations fc also not weH suited for 
expressing the behaviour of computations involving the production of stream structures. Recall 
from Chapter Two that streams are produced by tail recursive functions in a demand driven 
fashion. The unique instruction in a stream producer which, allows the, lazy evaluation of a 
stream is the suspension operator. The backup and n^v^ry^lgorUhins as currently defined are 
not capable of modeling the kind of program behaviour exhibited by stream producers for 
reasons discussed below. 



66 



FURTHER ENHANCEMENTS 



§4.4 



Steps in the computation of TailExample, with initial arguments: m = 1 n = 1 and f (x) = x + 1. 
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Figure 17: Handling Tail Recursion 
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4.4.2.1 Rationale 

To see why our current method is insufficient, consider the structure of the computation 
tree produced by the backup algorithm (as currently defin*d>4©F*stream producing function. 
Since such a function is taU recursive, its associated activation would be represented by a 
tauapplu ADE. The return link in a stream producer is used to connect together successive 
elements in a stream. When a new activation of a stream producer is Initiated, the return link 
which is passed to this new activation is the uid of the last stream element Thus, when the new 
stream element is produced, the field previously contaming the suspension in the last stream 
element would now reference this new element The new etement « turn, would either be a 
record whose second field is a suspension to the vnmmm instrection in the current 
activation or the value null denoting the empty stream. 

Now, consider the behaviour of the computation tree If theSTREAMTAiL instruction were 
to be treated as being identical to the tailapply operator. Under this assumption, our backup 
algorithm would preserve the argument record for each activation of the stream producer 
function initiated, in accordance with the description of tail recursion given above. Notice that 
because a stream activation only executes a return whin no more tail recursive calls are 
necessary, the only result value that would be preserved on tile backup store would be the last 
stream element produced. Intermediate elements which are constructed usmg the set and 
setsusp operators would not be maintained on backup ^.pkmovtr, recording only the 
argument record of the tail recursive activation for a stream producer would not be sufficient to 
restore the rest of the stream because the return link ft* each activation ^isjljf^t Recall from 
Chapter two that the return link passed to an activation of a s|ream pioducer is actually the 
second field of the record representing the last stream etement created b^ this producer. Thus, 
the return link of each call to the stream producer would be different 

This analysis indicates that the current design of the backup and recovery algorithms suffer 
from two drawbacks wim resr^ to me handling of s^ 

activations associated with a stream producer differ from each other in more than just their 
argument records, we need to maintain more information about die activation on backup store. 
The extra information which needs to be recorded must obviously include the new stream 
element produced. The second drawback is the inability of' uie backup algorithms to 
incrementally construct a data structure on the backup store. When a stream element is created. 
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it does not define the entire stream but represents only one element in the stream. Because 
streams are created in a demand driven manner, whenever a new stream element is created, there 
is also an activation associated with it whose slate is relevant to the backup system. The 
suspension signals an instruction in tiiis activation to initiate production of the next stream 
element Of most importance to the backup system is the argument record held by the 
streamtail instruction which instantiates the next stream activation. The backup system must 
record this information if it is to properly restore the state of the system. If die argument record 
is not copied, then mere would be no way for the recovery system to generate any further stream 
elements beyond that which has been copied onto the backup store. We discuss the 
ramifications of this requirement below. 

The instruction responsible for setting the suspension in the stream is the SETSUSP 
instruction, The argument to SETSUSP is the record representing the new stream element We 
see that one means of noting the production of new stream dements, therefore, is to alter the 
behaviour of the SETSUSP instruction. The SETSUSP instruction, m addition to setting a 
suspension in me new stream element, also initiates the transfer of the stream element onto the 
backup store. Of course, the value of the stream may be an early completion structure in which 
case it will be the responsibility of the SET instruction to perform the actual transfer. 

The instruction responsible for initiating a new activation of the stream producer is the 
streamtail instruction. The main operand to this instruction of interest to us is the argument 
record that is used to initiate the new activation. Recording the argument record serves a 
different purpose from its use in normal tail recursive activations. For streams, recording the 
argument record of the streamtail operator is essential to restoring the state of the stream 
producer activation to allow further generation of stream elements after the the recovery 
procedures complete. 

The advantage in altering the behaviour of the setsusp instruction to initiate the copying 
of the stream element instead of the streamtail instruction is that the transfer of the stream 
element can take place before a demand is made for the next element If we choose to record the 
creation of stream elements by making the STREAMTAIL instruction copy its return link structure, 
we would need to wait for the next demand to be made (since that it is when the streamtail 
instruction fires) before the copy operation of the current stream element can be started. 
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Because stream elements are produced to a demand driven fashion, if the evaluation of a 
bind expression yields a stream, the value field in <name, value> pair bound in the backup 
environment will contain a single element initially, namely the first element in the stream. As 
more elements of the stream are produced, they are ad ded onto the backup image and are 
considered as part of the stream Image in the backup environment * 

4.4.2.2 Implementation 

To monitor a stream producer, we introduce a new type of activation descriptor called a 
stream ADE. A strewn ADE is similar to a toikppUj descriptor in that both maintain 
information about a function activation other than just its return value. The stream ADE, 
however, in addition to containing the argument inic^d for the next stream activation to be 
initiated, also contains the stream element result of its associated stream producer activation. 
These stream elements are linked together on the backup hwprBming the recovery process, the 
backup stream image is fifst restored. The recovery system then Constructs; a /skeleton of the 
activation of the stream producer. This skeietonis used toWtfe^foduction of the next stream 
element when the next demand is made. The only instruction that can be enabled in this 
activation is the streamtail operator whose argument record is taken from the backup store 
and whose return link is the address of the last stream element. The suspension field in this 
element is set to the address of the streamtail instruction. The reason for storing the 
argument recottf is to set up the skeleton activation to siiflioil me demand driven execution 
mechanism for streams. When the recovery procedure completes, a subset of the stream image 
recorded on backup store will be restored Let <x£ ^.a^rii streaih elements recorded on 
backup store. Then the recovery system restores the first j elements, / £ n, where x, is the 
greatest element for whfcrr the witumifaAnc : r&mrT* lllbikfe^^tifiyy^alW iS&Lt^«ijr^uri element has 
been preserved; A skeleton iafvatibin is con^ru&etf 3 tg ! yf§i ife^^ element when the 
demand for it is made: Durmg me corisWictio* drWMrilfitffai^ records 

may be removed from the stream image when ttftom & MMaMM:^ wfll not be heeded 
during the rece^eiy prbccssi -' J - ■■"'■< ''^-^v' 4»^ sr* .^iv- 

When the suspension field in fhejast element ,$p 4he b^uo^jmags is accessed during 
recovery, the streamtail instruction in the skeleton 4tfg^pg^,$* ^ting thenext 
activation of the producer to produce they> 1 th elemenL We illustrate this process in Fig. 18. 

To summarize, unlike all the other structures wc have examined, there are two operators 
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Figure 18: Reconstruction of « Stream Structure 



which are responsible foe maintaining a corisi^( u^^ s pja§^^ig^ ThrSETSUSP operator is 
responsible for initiating the transfer of the $e w. stream element onto; the fednpe 40m., The Mnk 
field of the stream elements in th| backup image is u^4^ by : U^ STREAMTAiL operator when 
it executes. The streamtajju instruction is also responsible for copying the arguaaent record to 
the backup store. The backup system should treat the argurueot record passed to the 
streamtail instruction in an activation and the stream element created within that activation 
collectively to ensure that a correct stream i Image 1 IsisreserVeiJi Tie implementation of this 
<value, argument rccordis discussed in me next sectioW 1 



In the next section, we formalize the backup algorithms outlined informally above. Our 
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Formal model is an extension of the one given in Chapter twa'and mainly involves altering the 
definitions of those base language instructions responsible for the creation and updating of 
ADFs on the backup store. 

4.5 A Formal Model of Backup and Recovery 

Beyond the need for modifying the behaviour of some of the base language instructions, a 
formal model of Vim augmented with the backup and recovery algorithms must also have some 
concept of failure. A failure should be that special state which caiises the system to invoke the 
recovery procedures. The modified operation of the base language Instructions cHffe^ir&m their 
counterparts in Chapter two in that their execution results in a new backup state a$ well as in a 
new Vim state being constructed. 

In the following presentation, we shall be using the same notion as in Chapter two. 

4.5.1 The Backup State 

Formally, the Vim system is now treated as a four-tuple: 

Vim = ^<5A«?/7,//i/e/p,BackupState,ViiiiState> where 

VhnSUte = <Act X H X OS X Env> U (f ailed} 

BackupStare = <Log X BHeap X BEnv> 



The Act,H,EIS3nd Eav components in ths Ykp$^g^, j$i^ $^jdefinitipn* in model 
Ml. Notice that the VimStatf, .in petition to ^flfcjhe same ^p^nt^ic Ml also 
contains the special value, failed. A failure in the system is modeled by having the Vim$tate 
take on the value fatted. All information found in the current YJ^State is "lost" if the Vintftate 
has the fatted value. The domain BackupState is defined asa three-tuple where the first 
component in the tuple, Log represents the command td»ttf#f*?dlati& 5^ cotimiatids 5 , the 
second component, BHeap, denotes the set of |to$|ure v^L^^b^e backup procedures, 
and the third component BEnv contains all <namcmlue> bindings preserved by the backup 
utility. These values constitute the quiescent data in the system! T^I'dormunequatfon for BEnv 
is the same as that for the environment component in the Vim state, namely, 

Recall that volatile commands arc those commands whose results have either not been produced or have not been 
copied onto die kicktip store. 



72 A FORMAL MODEL OF BACKUP AND RECOVERY §4.5 

BEnv = Name -» (U H U Scalar) 

The command log, as was described in Chapter three, is used to hold the record of all currently 
volatile Shell commands input to the system. By safely recording these commands, we are 
guaranteed of being able to restore the correct system state, The la$ is f t^o-tupJe, consisting of 
a function mapping from natural numbers to log entries and a size component indicating the size 
of the log. New log entries are appended to the end of the log. 
Log = <N -f LogEntry> X N 

LogEntry = <Command X U B > 

U B = the set of uid's used for computation records. 

As new Shell commands are input to the system, the text of the command is copied by the 
backup procedures onto the log. this text corresponds to the Command component of the 
Logentry. The second component in a log entry is a reference to the cojnpjj|a#pn tree associated 
with this computation. This computation tree will reside on the backup heap. 

Every element on the backup heap, BHeap, hasjui associated uicL There are three types of 
elements on the heap: structures which are normal Vim structures discussed previously, 
activation descriptor entries, and stream coorchnacof records 1 ! that J are used t$ package 
information about a stream activation. We discuss^e*roJe«oC<|h* §|re^ coordinator^ record in 
greater detail when presenting the operation of the suspension operator later in this chapter. 
Every ADEtm the backup heap will e^f b% fetere^k^T^^ computation 

tree or will be referenced from a command tog entry Tf it is di^Triitiai ADE in the computation 
tree. ,v ""' 

BHeap = (U,, u Ug) - (ST U Ade U StrearaRec) 

StreamRec = <Val X AxgX Uak> 

Ade = <li A X AdcEntry X AdcTvpc X Result> 

Val, Arg = U H U Scalar u undef 

Link = U B U undef 

AdcEntry = (N -♦ U B U empty 



ii^^f-*^^-" 
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AdeType = {apply.tailapply, stream, value} 
Result = (U H U Scalar U undefi) 

An activation descriptor entry is a structure of four components. The first component is 
the uid of the activation being represented. As we show befo*i Ais field is used to identify the 
activation descriptor so that the result of the activation can be properly forwarded to the ADE 
when it becomes known. The second component; the AdeEntry, is sanction mapping from 
natural numbers to backup uids. If there am no entries Wthe AdeEntry, this component has 
value empty. The domain of the AdeEntry function is ft**tteir 48 instruction humbers in the 
corresponding activation which are ehher apply, tailapplY, o^ST*EA^OTAtt^>perations. The 
range denotes the uid's of the ADFs in the BHeap corresponding to these activations. Thus, if j 
was the instruction number in some activation a corresponding toift APPLY instruction, then 
AdeEntryQ) would be the uid of the ADE associated with the activation created by this APPLY 
instruction. Because the computation tree is pruned whenever a result of an activation is 
recorded, activation descriptor entries will have ihdr tfafefiftoy field set to the value empty 
indicating that there are no subordinate ADE* of th»%eti*tftlott and that the result of this 
activation has already been recorded. The third component fcfthe ADE contains me type of the 
descriptor. There are at least two types of ADES apply mm representing activations for 
which a result is not yet known; and value ^DFs wjijch c^u^ % In 

addition to these two types of descriptors, there ^.^^p^^fif^p^.fiot.pfil recursive 
activations and stream producers .which we described mti^e|pvto^ sections. The fourth field 
represents the result of the activation. It can either be a scalar or a uid which references the 
structure on the backup heap. All structures are associated wjdLpnly one uid. Thus, the uid's 
used to reference structures on the backup heap at»4he same .as those used to reference 
structures on the Vim heap. We shall use dot notation tojefer ty *WPW^ Pf an activation 
descriptor. 

When an ADE is initially constructed, there will be no value to place m its result field. 
Thus, this component is initially set to undef. When a result value is subsequently produced, it 
will replace the undefined element 
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4.5.2 Early Completion 

Early completion structures are represented in MR as follows: 
ECQ = 9(ECE) 
ECE = <U H X N> U {back} 

back is a special flag which indicates that this early completion structure is part of a structure 
which needs to be placed on the backup heap. This flag is placed in the queue by the Copy 
function. We present the definition of this function betow. When the SET operator replaces an 
early completion structure containing this flag with a value* it will check if the structure 
containing this field can be copied by determining if there are any more early completion 
elements in the structure. 

4.53 Auxiliary Functions 

As was the case with the heap in Ml, we have tm awulitjy functions defined on the 
backup heap, AddBHeapaa4 RemoveBHeap, which add and remove an element from the 
backup heap respectively. The definitions are omitted here but the reader may amply substitute 
BHeap for H in the definitions given for Addffeap andJiemo^Heapm Chapter two. 

There are also two functions defined on the command tog, AddLog and RemoveLog. 
AddLogadte a new log entry to the end of the log an^J?^ove£og entry removes a defined entry 
on the tog. In addition, we also define two functions over AdeEntries to replace and remove 
elements from a given AdeEntry. 



AddLog\ Log x UgEafry -* Log 

Function AddLog{Log t Logen'try) 

let <LogVal, size> = Log 

LogVtd'im) = LogvaAm) if m * size+l 
- LogentiyWm = size + 1, 
size+l 
in 

<LogVal\ size+ 1> 
endlct 
endfun 



»4^+« «*■■&? <&&* J& *■*-**»;'-* -*K<*"$t&~y tttde^f^ig^i^F^p-^J^-r + . 
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RemoveLog: Log X N— ♦ Log 

Function RemoveLog (Log, n) 

tet LogKat, size = Log 

LogValXrn)= LogVijt(m)ttm* n 
= undefMm- n 
in 

<LogVar,stie> 
endlet 
endfun 



NewAdeEntry. AdeEntry X U A X N -♦ AdeEntry 
Function NewAdeEntry(AdeEntry,u, n) 
let AdeEntryXm) = AdeEntryXm) Um* n 

- uYtm^A 
in' ' "-'>'* 

AdeEntry' 
endlet 
endfun 



R AdeEntry. AdeEntry x N -» AdeEntry 

Function RAde En try{ AdeEntry, n) 
let AdeEntryXm) = AdeVaUm) \Im*n 

in 

AdeEntry' 
endlet 
endfun 



4.53.1 The Copy Operation 

Before describing the Copy operation, let us ftm nJview iw abstract representation of a 
structure on the Vim heapr the heaj> Is n^de^^a'dm^iu^jii which every node is 
labeled with a unique identifier. Nodes on the heap correspond to structures in our system. This 
representation allows for structures to be components of other structures g.& an array of 
records. In our discussion, whenever we refer to a Vim structure, we also include this to mean 
all of the component structures which this structure references unless we explicitly state 



^'■■;*rw^*?£?%H; 
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otherwise. Thus, when a Vim structure is to be copied to the backup heap, it is also necessary 
that all component structures be transferred as well 

In order to preserve information found on the Vim state, it-is necessary to have a function 
which can transfer data from the heap and environment components of the VimState to their 
respective counterparts in the BackupState. This Copy function fe given below: 



Copy : (U H U Scalar) X H X BHeap -» H X BHeap 

Function Copy (Val,H,BHeap) 

let NewH, NewBH = 
if Ka/€Scalars 
then H,BHeap 
else let 

Re/Struct = {u | 3 m € N, R € Record s>L 
H( Vat) € Rec A H(Vat) (m) = «} 

u v tu u k = elements of Re/Struct 

ECStmct = {u € U H | H(u) € ECQ A Step{Val) = «} 

NewH', NewBH' = 
if ECS truct = {} 

then tt Re/Struct * {} 
then Copy{u v 

(Copyiw,,^ 
{Copy\u^HJHeop)J) 
else H, BHeap 
endif 





else AddBacHH, ECStmct), BHeap 
endif 
in 
NewH', 

if ECStmct ={} 
then AddBMeapKNmBH', YuliHiVa® 
eke AddBHeapiBHeap.val, ({WPCO&ut})) 
endtet 


in 

NewH. 
NewBH 
endtet 
cndfun 
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The Copy function takes as input the ukT of the structure to be copied and first determines 
if there are any early completion elements in the structure. If there are, the structure is not 
copied; instead, those fields which are early completion queues are pigmented with the special 
flag back. The function, AddBack, takes as arguments the torrent heap and the set of uid's 
which reference early completion elements in the s^cture. It returns a new heap in which the 
flag, back, has been added to each of these early completion itructures. Determining if there are 
any early completion elements in the stracture re^inres that all component structures be 
examined to see if they reference any suck element* Jlie function Step takes as input the uid of 
the top level structure and returns the set of all uid's referenced from any substructure 
referenced from it that is associated with an early completion structure on the heap. 

If there are no early completion elements m the structure, then it is transferred to the 
backup heap. Since a structure fray reference many substructures, the Copy function copies all 
substructures referenced from the structure by recursively calling itself. A structure if fully 
copied only when it and all substructures it references have been placed on the backup heap. 
While the Copy function is easily expressed in i our abstract model, it is significantiy more 
complex in the actual implementation. We address the implementation problems in the next 
chapter. 

4.5.4 The Shell 

The command log contains the text of the shell comand input and the name to which the 
result of evaluating this command should be bound This information is used by the recovery 
system which reinterprets the command text The cb%ujation record associated with the log 
entry is used to avoid unneccesarily reexecuting operations whose results have already been 
recorded. Maintaining this information requires modifying the operation of the shej| The shell 
must now. in addition to the stream of shell commands and curreqt JfimStoi^ateoMke as input 
the current BackupState. It returns a new VimStqte resulting' /^$e evaluation of these 
commands and a new BackupState which contain? a new k$ ent*y jof every command input 

'" •"* j 

Shell: Session x VimState X BackupState -» VimState X BackupState 

Function Shell (Session. VimState. BackupState) 

let <Acl H. EJS. En\i> = VimState 
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<Log, BHeap, BEnv> - BackupState 
NewState - 

if ChooseToExecute ( VimState, Session) 

then ShelRSession, Execute (VimState, ChoosefEIS)), BackupState) 
e)stti empty(Session) 

then VimState, BackupState 
else let c, = fk$t(Sessidh) 
in HCj.C * DELETE 

then <Act, H, EJS, &elgnviEny,Name)>, BackupState 
ebelfc 1 .C'= bind 

then let % command is BfND 



FA = TranslateCc^ 
u- A = ne^uid tt6m U A 
Act' = AddAcKAct, u F/f FA) 
NewEIS *£/$&%&*& *HF^i).opcnt = 
AFAtyStyfW^M 



Bj - anewuidinUg 



BHeap' = AddBHeapiBHeap,u x ,NewAde) 
NewLog - Addla^Lsg,Logmtry) 



State, v = Interstate, Choke{NewElS)) 
<Act\ H', EIS', Env> = VimState' 
Env' - AddtoEnv(Env,c v name, v) 
in 
<Act',H , ,Em',EnyX 
<NewLcg, BHmp\BEnv> 
endlet 
endif 
endlet 
endif 

in 
if empty (Session) 

then if £/£>{} 

then Shell (Session, ExecuteiNewstateXhoiceiEfSy)) 
else Newstate, NewBackupState 
endif 
eke Shell (rest{Session), Newstate, New BackupState) 
endif 

endlet 
endfun 
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The new log entry constructed by the shell contains the text of the command input and a 
reference to the root of the computation tree to be alssoeiatia^tff tflis computation. The uid 
field in the root activation descriptor entry contains the 1 uid of toWactrvatioh being instantiated. 
Its AdeEntry is undefhted since no functions have fcelsh afplieo 4 from tiiis activtatibn yet The tog 
can be thought 6f as an array of log entries. The stlef tfrJc^lheiog oy adding the new log 
entry to some currently undefined index, Note Jhat ti^^ii^^oi^^vokecl after the tog 
entry has been recorded. The shell acknowledges a shell input by demanding the next command 
in the input stream oriffwTien flic current shell' &smTmiric1 iat been mfted on the backup store. 
This guarantees that no processing will be done on a computation which cannot ^e recovered 
from failure since the text of the command is usettlJy fte m&i&j f j^ocedure^ to reexecute the 
command. The two functions which are responsible ft?* updaJiii^the ^fiMmiient DelEnv and 
AddEnv must also be modified. The 4>nWn¥ is reSponslbte'M removing an environment entry 
from the current environment Since environment biding* are aisp part of the backup state, 
DelEnv must remove the entry from the BEnv as welL 

4.5.4.1 Removing Log Entries 

The AddEnv function irresponsible for adding aiiew <name,valu«> binding to the current 
environment The backup system «a«^kw mJbRnation In the ftJrtn 6H computation record 
about the computation that produced the value. This computation record nee^jje kept on the 
backup store only so long as the binding was not placed in the image of the envircrtiNfftt kept on 
the backup state. Once the binding is placed in the backup state environment and the result 
value is fully defirw^ the coaptation ^ can 

be removed from.the bjcjtupiiea^. I^may bethf case, ha^yer, tiw the result value contains 
early cojnr^etipns^ciu^p^ befofc^e^jonjiyhj^ 

component in the bkiding in this cgse isnotcof^ 

the set operator p remove, %eompjuja^ If 

there are.no early cranptetion sjructuj^ > ^^ peaces the 

binding on the backup environment and, .removes, the lqg,«nu? fpc this compilation as well. 
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4 .5 J The Interpreter 

Because base language instructions will be now be operating on the backup state as well as 
the Vim state, it is necessary to alter the behaviour of the interpreter. The interpreter is now a 
state transition function from a VimState, a BackupState, and an enabled instruction to a new 
VimState a. new BackupState and a result value. Its defu^tiQQ jsgiven below: 



Interp: VtaiState X BackupState X O - VimStaie X BackupState X (U H U Scalar) 
Function Interp ( VimState, BackupState, <«,<>) %<u,£> is a&enabled instruction 

let 

<Act,H,EIS,Env> - VimState 
FA = Act(u) 

<Log,BHeap{BEm> * BackupState 
NewVimsW&NwBwtelSU&^EmwttfVmiStitiM^ 
NewVimstate' = Failure (NewVimstate) 
<Act',H',E/$r, Ertf>^ NewVtmitate' 
in 

if faileaXNewVimstate') 

then Recover&BackupState) 

elseif FA(f).opcode = terminate 
then NewTim&ieWtvi&dekupS^ 

else Inte?p{NewViniStaie&ewWtip$MtJM 
endif 
endlet 
enttiun 



The Failure function models the introduction of a fatted state in the system. It returns 
either the state failed or the state passed to it as rnptit The function fatted returns true if the 
new state is a failed state and false otherwise. When jfeftafteturns true, the Interpreter invokes 
the recovery procedures to restore the system to a correct estate;' The model of failure given here 
is, of course, simpHstic insofar as it assumes that a faifure does not occur during the middle of 
instruction execution. In the actual rniptementatfon tff the system, care must be taken to 
guarantee that copy operations on the backup store art performed dtomically. The model given 
here, however, is convenient for expressing the salient aspects of the backup algorithms, 
abstracting low level details such as preserving atomicity of copy operations. These issues are 
addressed in the next chapter. 
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4.5.6 Function Application 

In this section, we present the modified operational semtffitkSbf twoof the base language 
instructions responsible for function application in our syste% the apply and the tailapply 
instructions. Htt > 

4.5.6.1 The Apply Instruction 

The effect of executing the apply operator is to augment the number of actuations in the 
VimState. The addition of a new activation is reflected on the backup heap' by adding a new 
activation descriptor entry for this new activation and creating a new Adegmm Unction in the 
ADE of the currently executing activation. This new function maps from offset to uid where 
offset is the instruction number of the apply operator and uid is\he unique^ igejatifier of the 
ADE representing the new activation on the backup heap. The new ADE will coBtafir in its uid 
field, the uid of the new activation; its AdeEhtry field is set "to empty, and its type field is 
initialized tq appty WWW that thj^activa£?n. was instantiated t* an apply instruction. 
Since the result of thjs Ration is.not yet known, its ResuJt^sttto undefined. 

if /.opcode = apply den 
.Hit ■■ 

C = /.opl, 
arg= f,op2, 

<u f free> = H(Q, 

u' = a new uid from IL , 

«"= anewultiftomtf., ' 

Act' m AddAcHAci.u'Mu A 

H' = AMHeai!(Hj*"M<&iQ&M# m It 

Act", NtwEis' '« 
SendToDest 
(SerutToDest 
(SendTbDat 
(Act'. NewEis* u ', <uncon4, 1. opt>, Q 
u\ <uncorut, 2. opt >, arg) 

u^ = a newuid chosen from U» . 
newade = <u\ empty, {apply} unde/>. 
&Heap x « AddBtt^meaph^newade); 
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BHeap 2 - if 3 u b s.L Bheap (u^ = <u Fj ,, AdeEntry, Type, Resulf>, 

then tet AdeEntry' s Nmj^^trj^B^&^u^AdeEntry, u^i), 
Update Ade = <u FA , AdeEntry', Type, ResulO, 
in 

AddBHeap(BHeap v u^UpdateAde) 
endlet 
eteeBHeap 
endif 
in 
<4c/". 
#', 

NewEls', 
Env>, 

<Log,BHeap r BEm> 
endlet 
endif 



It is possible that there is nb computation record on the backup lieap that contains the 
activation descriptor for the activation in which the aMY instruction Is found. This may 
happen if the result of the computation has already been bgpd in the backup environment 
before the apply executes. In this case, there is no change to the backup state. I^there is an 
activation descriptor corresponding to the current activation, its AdeEntry ^updated to reference 
this new ADE. 

4.5.6.2 The TailApply Instruction 

The TAILAPPLY operator is also modified to monitor the progress of tail recursive 
functions. If a tailapply operation executes as part of a tail recursive activation, we copy the 
argument record passed as the second input to the instruction to the backup heap, replacing die 
old argument record found in the activation descriptor entry. As we mentioned earlier, doing 
this substantially reduces the time needed to reexecute a tail recursive njrictton. Copying the 
argument record becomes a non-trivial issue, however, because of the presence of early 
completion structures. If the argument record to be copied contains early completion elements, 
the copy operation can only take place after all such efan|flts haye been set The old argument 
record on backup heap and the AdeEntry component cannot be replaced until the new record is 
fully copied. To ensure that this restriction is observed, it is necessary to create a new ADE 
corresponding to this new activation. We now need to link ih© old jADE and the new ADE 
together. Since the idea of noting the argument record on the backup heap is to avoid 
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reexecution of prior tail recursive calls, we interpose the new ADE between the activation 
descriptor corresponding to the current activation and its parent ADE. The parent ADE 
represents the activation in which the initial call to the tail recursive function was made. In this 
sense, the computation tree btrilt from tail recursive activations isconstfucted "bottom-up", whh 
old tail recursive ,4Z>Fsl>eing pushed down the tree and new ADEs&mg fitted in between its 
caller and the original caller of the function. We illustrate this process in Fig. 19. When an 
argument record is copied, we set the AdeEntry field ! in th$ descriptor to empty, effectively 
discarding prior ADE% associated with this function. Thus, once an argument record is fully 
copied, all previous ADFs of this tail recursive function areremoved from the backup state since 
the link connecting these activation descriptors with the rest of the computation tree is severed. 
The formal definition of the modified tailapply instruction is given below: 



if /.opcode = TAiLAPPLY then 
let 
C = /.opl, 
arg = /.op2 
dest = /.op3 

<u f /ree> = H(Q 

u - a new uid from U A , 
Act' = AddActCAcuW, H(u) 
Act", NewEis' = r 

SendToDest 
(SendToDest 
(SendToDest 

(Act\NewEis,M\<\mGmfU..opy\Q 
u\<unamd£apt>,arg) 
w',<uncotuU.opi>, dest) 

Act"', NewEis" = SendSignal(Act''J*e^k'm F t^stimions) 

U new ~ new u 'd fo° m ^B 

BHeap l — if 3 u^u^k &t BHeapiu^ <u a .AdeEtitty,Type,Result> 

then AddBUeaHMHt^M^u^ v. £% t.-m tA 

MewAdeEntotAdeEntry.uAWype.Resulfr) 
dsxBHeap ^ 

endif 
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After first tail recursive call 



Second recursive call 




► J notcopied j 




After argument record of second call copied 



Ul 


empty 


taMpply 






Figure 19: Recording Tail Recursive Functions 



H\BHeap 2 = \\BHeap x = BHeap 
then H, BHeap 
eke CapyiatgtfiBHmp*) 
endif 



/?#eap 3 = if BHeap 2 (arg) * notcopied 

then MBMeapi BHeap 2 M n ^<u r jtmpt\tfaUapphj.arg>) 
elseti3u^&kBNmpfrJjM=Urt 

then let AdcEntty'-MtwAi^£mfyi{f,it ade .i) 
in . 

AddBHeap(BHeap 2 M mw <u\d&Etti*y\ta\lapplyMrg>) 
endict 
else BHeap. 
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endif 
in 
<Act"\ 
H\ 

NewEis" 
Env>, 

^og.BHeapj.BEnvy 
endiet 
endif 
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The backup heap is modified in three steps by the tailapply operator. In the first step, 
the ADE of the activation which initially ins^tW^^ij i^ursW$ function is modified to 
have its AdeEntry field inference the activation deacriptot CA«mj«««f for this new application. 
This new descriptor has its AdeEntry field initialized to reference ^4DE of its caller. If no 
such ADE exists, then no change is made. In the next step, the argument record is placed on the 
backup heap. If the copy operation was siccessfu^ then, in ^ J^d'step, the AdeEntry field of 
the newly created descriptor is set to empty since th© recovery ficedure can use the argument 
record directly during the restoration of the system state. If the sfcture was not copied, then a 
reference in the AdeEntry component of|ie new activation U setyto i$ caller Me. This allows us 
to still reference previous activations «£*he tail necuraive function until the argument record is 
finally copied. '"•'■' v \' 3 

4.5.7 The Return Operator 

One of the important features of our tadu^tfgorithm is that results of activations are 
recorded on the computation tree. Tmsjs the prtn^means of reducing reexecution time of 
volatile shell commands. The RETURN Qperatp/ transmits the result of an activation to the 
backup heap. The return value,. if it is a Vim s&itetere will, in most cases, contain early 
completion elements. The SET operator is responsible, in thes4|r|ms|ances, for ensuring that 
the structure does finally get copied. If the return value is copied, then aJI ADFs referenced 
from this descriptor are removed from the backup state. This has the effect of pruning the 
computation tree. Because ADEs can be removed in this manner, it may be the case that when 
the return instruction executes, there maybe no ADE associated with its activation since a 
return instruction in a parent activation may have placed its result or>|he bjaekup heap. In this 
case, no action on the backup state is required by the instruction. The modified definition of the 
instruction is given below: 
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if opcode = return then 
let 

DL = #(/.opl) % the list of return addresses 
u f = Dl/1) % uid of the calling activation 

targets = GetDest{H,DL(2)) 

Val = /.op2, % the value to be returned 

Act', NewEis' = SendValue{Act,NewEisM^ targets, nai) 
Act", NewEis", = SendSignaKAct',NewEis',u FA ,destinations) 

Ade = <u F/i ,AdeEntry,TypeJlesult> 
BHeap 1 = ti3u b ±t BHeap\up^ Ade 

the* AddBHeapiBHeap.u^UpjAdeEntry.value, Vat>) 

else 5//eap 
endif 

H\BHeap 2 = KValZl) H 

then CopM Val,H\BHeap') 
efei H,BHeap x 
endif 

H",BHeap 3 = if 3 u ft if. BffedpJuJ = /*<fe 

then if BHeapAVaf) = notoopUd. 
then H',BHeap 2 
else let 

/??f = {/i| AdeEntry(n) is defined} 
/ij./i^...,^ = A: elements of fl^V 
AdeEntry' = RAdeEntry(... 

m^aWhtr^dmn^npJ:h k ))) 
NewAde = <u FA ,AdeEntfy:mkm t Va£> 
in 

H\AddBHeap(BHeap 2 ,UpNewAde) 
endtet 
endif 
e\serf,BHeap 
endif 
in 

/r, 

NewEis" 
Env>, 

<Log.BHeapyBEnv> 
endlet 
endif 
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The backup heap is updated in three steps by the RETURN instruction. The first step 
determines if an activation descriptor for the current activation exists** if this activation has an 
ADE which is part of some computation tree. If so, tfte type of this activation descriptor is 
changed to value indicating that a result has been created and the return value fc Writteh into the 
result field of that A&E. If the return value is a structure, then we initiate a Copy operation to 
place this structure cm the backup heap. If the structure is fufly copied because ftcontinns no 
early comrdirttott elements ortftrwrtsuttwas 

descriptor are removed. from the backup, heap, effectively -jmuung-^he computation tree as 
discussed earlier. If, on the other hand, the result was a structure contains early completion 
structures, it is the responsibility of the SET ii^wotidft to perform Jhe pruning of the 
computation tree. 

4.5.8 Stream Operations 

There are two operations which manipulate stream activation descriptors on backup heap: 
the setsusp instruction and the streamtail instruction. These operators are responsible for 
recording the creation of new stream e^nente and^jnoj^ argument records of activations to 
allow the recovery procedures to reconstruct the stream image ami to permit demand driven 
evaluation of those elements not recorded. 

4.5.8.1 The Suspension Operator 

As we had described earlier, the setsusp operator needs to be altered to record the 
production of stream dements on the backup heap. Centtal to our a%orithm is the use of a 
stream coordinator record which contains the element produced by a stream producer activation 
and the argument record to the STREAMJAIL instru^ 

producer. When a new stream activation is i n s um ia u d, a iww,4££ is created for it tmhkethe 
case with the apply operator, however, ADf s created by,tae $TNeA*iTAtt, instruction are 
accessed through the sireamxaordinator record. Theiink field in the coordinator-record is used 
to link together all stream activation descriptors in a chain; limiting the construction of the 
stream on the VimState. The setsusp instruction is responsible for irtitiajly creating the 
coordinator record and for linking the elements in the backup stream* fcttage. When the 
streamtail instruction executes, it initiates the copy operation for the argument record of the 
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new activation being instantiated. In addition, it sets the link field in the coordinator record of 
its corresponding ADE to that of the new stream producer activation. 

The SETSUSP operator takes as input the record representing the new stream element It 
updates the ADE associated with this activation to type stream and constructs a new stream 
coordinator which is referenced from the Result of this descriptor. It then initiates the copy 
operation for this stream element Each new invocation of the stream producer causes anew 
ADE to be constructed, with the link field in the coordinator of the previous activation set to this 
new ADE. In Fig. 20, we illustrate the effectof theSEfSUSP instruction on the backup heap. 
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apply 


undtf 



Computation tree 
rooted at this activation^ 




After a SETSUSP operator In activation with uidu executes, 
andstrean element recorded. 
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/Computation tree^v 
/ rooted at dris activations^ 




Flgim 20: The EfTect ofthe setsusf Instruction on the Backup Heap 



The formal definition of the SETSUSP operator is given below: 

if /.opcode = setsusp then 
let 

U = /.Op! 
R = H(u) 
f= /op2 
/ = Aop3 



>.-***r-^-*rv*t*- • -« •■■- •* :--.«• ■ »**g~ ^-'-ya^ 
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= MakeSusj{<u EA J>) otherwise 
st record = <*{!), tf/«^y^l$rgff^ lenown yet 

else # Jftrflsp 
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else H.BHcap 






in 

if^(/)€ECQAfe(^ = 5 
■ •■ tlwi<L4cf, •-' p -- ! '"--- ■= ^"" ; 

7Ve»v£^' 

else letaer^ft^ &^ 

■;.■■..■■'.!»:-• -,.r-'..o'-,j i':,- ! \r,--i—-i J!ATM/.iiST?i ail! oJ'b5«*a !:m;y-i vk-:':.-. :: 

■ NtwEti", ■- -vr-^i :■;*■ ---..,-d qi.jbso ■;;■ :-; ',./,: -. : ■•.!;; 

<Log, BHeapj, BEnv> 

endlet 



There are three major tasks that the SETSUSP operator is responsible for in iM update of 

U . I*|0 i ~~ 

the backup heap. The first is the construction of the stream coordinate^ ftcor^ It creates a 
record, whose first component is the V«fce of tf* J htwVMitefl kream elWtem, $ntJ the second 
and third components arc given value, undef. This second components to hold Rvalue of the 
argument record but may only be defined when the streamtail instruction in this activation 
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executes. The third component holds the link to the ABE of Hie next activation of the stream 
producer and can also only be set when the streamtail operator executes. The second task is 
the copying of the value component of the stream record. Recall that our definition of a non- 
empty stream element is a record of two fields, the first being the element itself and the second 
holding the suspension to the rest of the stream. The backup facility need only copy the first 
element of the record since the link field to the next stream [element can be set by the recovery 
procedure when the stream is reconstructed The third major task that the operator performs is 
the updating of the activation descriptor entry associated with the stream producer. It changes 
the type field of the ADE to stream and updates ^ ifce JUw## reference the new stream 
coordinator record. We next examine how the streamtail instruction needs to be modified to 
maintain a consistent and up-to-date image of stream production. 

4.5.8.2 The StreamTail Operator 

The streamtail instruction operates in conjunction with fhe SETSUSP operator in 
maintaining the stream coordinator records and activation descriptors. When this instruction 
executes, it initiates the transfer of the argument record passed as its second argument onto the 
backup heap. The stream coordinator corresponding to this activation ^updated to reference 
this argument record. If both the value field and the argument record are fully copied, then all 
subordinate ADFs can be removed from the backup heap. In titis sense* the stream value and 
argument record passed to the streamtail instruction constitute the result value of this 
activation and just as the computation tree is pruned by the return instruction when the return 
value is placed on the backup heap, we perform the same action here when both the stream value 
and argument record are safely recorded. Like the tailapply instruction, STREAMTAIL is also 
responsible for creating a new activation descriptor. The type of this 4|)# is set to stream and 
the link field in the current stream activation record is set to the uid of this descriptor. We give 
the formal definition of the streamtail instruction below: 



if /.opcode = streamtail then 
let 
C=Aopl. 
arg = 7.op2 
dest = Aap3 % dest is a field in a stream record 



<u f free> = H(Q 
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w'= anew «id from U., 
Act ' = AdiAct(AcUu\H(u) 
Act", NewEis' « 
SendToDest 
(SendToDest 
(SendToDest ■ ■-,- ,;v 

Me/ \NewEis+ ^\<un^ru^9ft\> r Q 
u',<uncoruU,opl>, a/y) 



/Icf", Afew£/$" = SendSi^Art",NewEis\u^dfs^ 

newSfe = <u '.empty, stream, um/e/> 
£#«*/>! = AddBHeaABHmp,u newad ,nemd€) 



IT.BHeap* = If 3 k.xA BttmpSu^ = <u FA ,AdeEntry&nam,u> 

else H,BHeap 
en«* '""- .>;■<>.. 

tbca let 

Adetntry .* RAdtEntry(.„ 

> - ^kmmtAW^h^:!h^ 

in 
else H",BHeap 1 
in 

£/TV>, 

<Log.BHeap y B£nv> 
ei^ltet 
cmiif 



"sj^^^c; - * -. -i- s- . : 
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There are four steps that the STREAMTAIL instruction follows in updating the backup heap. 
The first step is the construction of the activation descriptor for tr^j^^ stream activation to be 
instantiated. In the next step, the instruction calls the Copy fune&Mi to transfer the argument 
record to the backup heap. In the-*W^Wv^J^^':^?i[^ii|& associated with this 
activation is updated to have its argument record rcfefencefthK^wture. Finally, the instruction 
checks to see if both the stream element and the argument record have been copied, and if so, 
removes the subordinate ADE*s frdm the backup heap. We have avoided describing the removal 
of argument records of old stream coordinatior records to Simplify the presentation. 

4.5.9 The Set Operator 

The operation of the set instruction is also m6tf^d loltod ihfermition on the backup 
heap. A structure containing an early coj»pie|qn eje^ argument record to 

a tailapply instruction or 4ny*fa4*tfc»e#i mMcbbeift^i activation. In section 4.3, we 
argued that such a structure should augment the lockup %^only when it and all of its 
substructures are fully defined ie when they contain no early completion structures. If the early 



completion queue in the <r-*nictUTe being set contafti the M*MjL vtim^tiil it means that this 



. . . . T)l* . . 
field is part of some structure whidt needs to 4m rope* Onto the backup heap. In fact, because 

elements on the heap <^bVstared, thestrti&ure containing this field may be a component of 

many structures, some oCiW^jpfBe^^^.^fp^^i^cklip heap. The SET instruction 

examines each of these structures to See rf there are aa^j&rly completion structures in them 

which still need to get set Those str^i^w|ijc^ are copied. 



The SET instructk»valso examine* those activation descriptors or stream coordinators that 
have a reference to any of those structures that become f^Befined because of its execution. If, 
for example, there is a uolue or tauapptu Ade whose ^JS^ferences a structure that 
becomes fully defined because of the set instruction, then, as was^lfejHe by the return and 
tailapply instructions, all references to activation descriptors from the AdeEnifycymponent of 
the associated ADE can be removed. Similarly, if the reference to a fully defined structure 
emanates from a stream coordinator, then the SET operator must check to see 5f the AdeEntry list 
of the associated stream activation descriptor can be reset usingihe criteria we discussed above. 
The instruction is also responsible for removing a log entry if the element being ! '&%% part of a 
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structure to be- bound in an environniMit We give the formal definition of the instruction 
below: 



if /.opcode = SET then 

let ?> "%,-,,. . 

u= /.opl 
R = H(u) 
/= /0P2 
x = /.0p3 

«'= #(*(/)) 

Vv€N 
iT(v) = tf(v)lfv#/ 

= x otherwise 

Act', NewEis' = SendSignaKAct,NewEis,u FA ,desiindtion# 
H'=NewHeap(H t u,R') 

Parent ={u£u r €Siepf$fi 

Hj.Uj «jt = * components of Parent 

duett Copy (u p 

t\&H',BH m 
entit 

then let Ref - {n | AdeEntryJn) is defined} 

m if Type = value V 7>pe = toUopfity 

.,• •fl^^^'TrWW^ "ilWwHPW^Ojvfn ..:.t.'..' 
then//' 

A<jd6lteafi(Wfeafi v ify<»p^4kEntryiTypeMj>) 

else 5//«v, 

ewdif u ' ; '' 

ebeif 7>/>e = streon* y ; >< 
then let <val.arg.Hnk> = BHeapfuJ 

mitfvul *7ioW»fH^V ; ^o«p(Hff/)^notco|He<t) 
A BHea^arg)* notcppied 
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endlet 
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endif 
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endif 
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endif 











Because objects on the heap can be shared, the record whose field^is being set may be a 
substructure of several objects. The set Parent denotes tfft rfid^fceMttrucita Recall that 
the auxiliary function Step when given the uid of ai^e|^mpie&n structure returns the set of 
all uids associated with structures which have *path o* the heap to this ec-structure. If the flag, 
bock, is found in the early completion queue, the inMcl^h'ca^e Copy function to attempt 
to copy all those structures whose uid's are in Parent. This function will only copy those 
structures which do not contain any other early a^0ebft^£n^" The backup Heap returned 
from the calls to Copy will c^t^^l^^^^^i^^ froin Parent which were able to 
be copied because they no k*g^ contained any ftHrtructures. 

If the activation descriptor^ referencing a copied strltofafe was of type value or tatlapptu, 
then the subtree rooted at the ADE is removed. If. on the other hand, this structure was 
referenced from a stream coordinator recerS, the Instruction removes the subtree of the 
corresponding stream activation descriptor if both rf»e value and argument field in the 
coordinator have been completely defined 

We also introduce one other auxiliary function; CneckLog. Some of the structures in the 
Parent set maybe values which are to be boun$yn the user environment. Among these 
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structures, there maybe some which are now fully defined as a result of executing the set 
operation and are placed c>n the the environment image on the 'backup state. These structures 
are the result vaJuesofaelh* computations. These computations a*e represented on the backup 
heap as computation records whk* are referenced rkim the entrks in the command tog. The 
function CheckLog removes these log entries and returns the new log. 

4.6 Summary 

In this chapter, we presented the backup and recovery algorithms for the Vim systenv Our 
attention focused on the representation and manipulation, of computation records, whfch, contain 
information about the progress of volatile commands in the system. A computation record is 
represented as a directed tree where nodes in the tree denote activations and edges signify 
caller/callee relationships. To update a computation record requires altering the semantics of 
the apply and return instructions. The apply operator adds a new node to the tree and the 
return operator replaces a node with the result value of its corresponding activation. 

A node in the computation tree is referred to as an activation descriptor entry and embodies 
information about an activation in some active computation. There are two bask; types of 
ABE'S: apply and value, the former used to denote an activation whose result value is not yet 
known and the latter designating an activation whose value has been recorded on the backup 
heap. 

The main disadvantage with this simple scheme is that it is not sufficiently expressive to 
handle early completion or stream structures and is inefficient when tail recursive functions are 
involved. Early completion is handled by requiring that all ec-elements in a structure be SET 
before the structure is copied. Ensuring that such a structure eventually does get copied 
necessitated the modification of the SET instruction to update the backup heap when the 
structure becomes fully defined. Tail recursive functions are represented on the backup heap 
using a special taUoppty ADE which holds the argument record of the tail recursive activation. 
We based the design of the tatlappiu: ADE on the observation that the recovery procedures 
could avoid executing intermediate activations of a tail recursive function if the argument 
records to the activations of the functions were recorded. During recovery, the function could be 
instantiated directly with the argument recorded found on the backup heap. Finally, we 
introduced a special descriptor to handle stream structures. A stream ADE contains a reference 
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to a stream coontimtor mord which embodiea information about a stream activation. In 
addition to storing the stream elenw^t produced iti ihit activation, we abo keep the argument 
record needed to produce the next stream element. Theconstrnction and maintenance of stream 
coordinator records required modi&itit the operation \ of aw SEfSUH' and SfRBAMTAlL 
instructions. 

The algorithms presented in this chapter were developed for a very abstract machine in 
which such issues as structure organization, guaranteeing atomicity of information transfer from 
memory to backup heap, ana memory management of the backup heap were not addressed In 
the nekt chapter, we dweuss these tissues. 






§5.0 97 

Chapterf ive 
Implementation Issues 

In this chapter, we are concerned with concrete problems that arise when we attempt to 
implement the backup algorithms presented in Chapter Four for the Vim system. These 
algorithms were described in the context of an abstract model MR which was used as a vehicle to 
rigorously describe their behaviour. Issues which were conveniently hidden in our model will 
now have to be more thoroughly addressed. In particukr, we will focus our attention on how the 
Copy function may be implemented. The complexity of the operation arises because of the 
representation of structures m our system. Guaranteeing that the oopy operation will always be 
Atomic b a non-trivial task given the storage representation for data structures that Vim uses. 
Section 5.3 concentrates on this issue. Section 54 describes a drnpte storage management policy 
for stable storage, 

5.1 The Copy Operation 

The copy operation, as described in Chapter Four, is responsible for copying a Vim 
structure object to the backup heap. Anjefficient impterrwmatton of this operation is necessary 
for any practical realization of our backup algorithms. In our 'tfbNRttMbdd^titecop> operation 
was treated as a fur£txHiwhk&traastn^ The caHers 

of this function were base language instructions that required result values or argument records 
to be placed on the backup heap. There irno c©rtcurreri#fbe*w«it the execution of the caller 
te. instructions and the C&fiy function in this model. An instruction tfcatcalis the Copy function 
must wait for the copy to be complete before it can continue execution. In an actual 
implementation, having the interpreter watt for the transfer of data from memory to stable 
storage before beginning execution of the next instruction would lead to intolerably slow 
performance. Examination of the formal description of these instructions, however, reveals no 
reason why the transfer of data to stable storage cannot proceed ill parallel with th& execution of 
other, instructions. Our approach to implementing die backup algorithms is to consolidate the 
"logic" for modifying the backup state into a backup system kernel that is responsible for 
updating the computation records on backup store and initiating the copy operation to perform 
data transfer to stable storage. Base language instructions invoke the kernel, passing as 
arguments, the type of operation to be performed Le. set. result, tattapply cic. and the necessary 
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operands Le. data structure to he copied. The iflstructions can then proceed with normal 
execution without having to wait for any results from the kernel. The responsibility of updating- 
the backup state would then be primarily in the ftafvfew of the backup system kernel. 
Overlapping execution of base language instructions with transfer of data to and from backup 
store makes the backup algorithms presented a much more attractive solution. If sufficient 
concurrency can be exploited in the base language programs, then we conjecture that updating 
stable storage should not cause major degradation in system performance. 

An important requirement of the Copy function is that it be atomic. In database literature, 
atomicity is a property of a transaction whose overall effect is altor-nothing: either all the 
changes made to the data by the transaction happen, or none of me changes happen. Thus, all 
transactions appear externally as indivisible operations. This requirement is essential to support 
recoverability of data after hardware failures occur. Atomicity in Vim is a property of an 
activation that refers to the point at which the effects of the activation are perceived by other 
executing activities in the system. It is necessary that the Copy function be atomic because the 
effect of its execution Le. the augmenting of backup state information, should be hiade visible to 
the recovery procedures only after the entire structure toes been completely copied to backup 
store. If a failure were to take place during the middle of a copy operation, and the function was 
not atomic the recovery system would see data in the backup state that does riot correspond with 
any data that was present in the Vim state at the time the failure toot place. Guaranteeing 
atomicity is a non-trivial issue for two reasons. First data structures in Vim may be arbitrarily 
complex; if the structure to be copied represented a node <*on the Vim heap, then the entire 
graph rooted at a must also be copied onto the backup heap- "Pie other complexity involved 
with the copy operation is due to the way data structures are represented in Vim. We discuss the 
storage organization of the Vim heap in the next section and then present our solution to 
guaranteeing atomicity for the operation. 

5.2 Storage Organization in Vim 

Vim structures have been thus far treated as monolithic entities that are created and 
manipulated as a single unit This representation was useful in the presentation of our backup 
and recovery algorithms but is far removed from the actual representation of structures in our 
system. The representation chosen for structures in Vim is influenced by the organization of 
physical memory and the desire to have only information needed by the computation be resident 
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in main memory. Vim is based on a hierachically organized physical memory consisting of main 
memory and disk in which information is brought^te mw*m<W only upon demand. To 
facilitate the transfer of information between memory and disk, the virtual address space is 
partitioned into a number of fixed size pages. The organization of the address space in Vim 
differs from conventional demand paged systems, however, in the page size chosen. To avoid 
the overhead of unnecessarily paging in unwanted information, 4he unit of storage allocation in 
Vim is a small page of 24-32 words, known as a chunk. Having a small page size is the primary 
means of exploiting parallelism in base language programs. In our data flow execution model, 
there is no dependency between any two enabled in$tni«i©BS:a9d thus, I/O service required by 
one instructiori does not prohibit the execution of the other in (fee interim period. By having a 
high level of concurrency of data transfer between the disk and main memory, the processing 
unit is seldom expected to be idle waiting for a pending I/O request to be serviced during 
program execution. It is expected that, in general, Aere will be enough enabled instructions 
during program execution to make disk access completely transparent 

Because of the small page size used, each chunk holds at most a single object Complex 
structures such as arrays and records are held in a number of chunks. The representation chosen 
for these structures should be sensitive to the applicative nature of the programming model by 
allowing information to be shared between structures whenever possible. One implementation 
well suited to our goal of efficient sharing is to represent VMstmctiUES ** k-ary trees of chunks 
with the leaves of the tree containing the elements of the stnkuija and the eternal nodes of the 
tree containing pointere to other <5hunW{l7J. Beeause^trueWrcan be complex, leaf chunks 
may hold uid's to other structures in addition to containing scaj$: values, The choice of a tree 
organization to represent chunk^aMows a high degree of sharing^© ^eaehieved. For example, to 
construct a new version of a structure differing from its predecessor in a single element the 
system need only construct a new path from the root of the new structure to the leaf chunk which 
is to hold the new element; all other elements which are sfif comltftn W bolh structures are still 
shared between them by having both structures use the same paths to thr common Jeaf chunks. 
The replace instruction briefly described in Chapter Two, for example, which returns a new 
version of a record differing from the old version in a single element operate^ m this? manaer. 

The address of an element in a structure is specified by a two-tuple, <uid. accesspath>. uid 
denotes the uid of the structure in the system. The accc^pdtfy¥th¥basc ft representation Of the 
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offset of the desired element in the structure. The length ot^tit access path for a given structure 
is the height of its Vim tree representation. 
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Abstractly, we view a chunk as a three tuple: 
Chunk = Cid X Header X Data 

Header = N -♦ N 

Data = Internal U Leaf 
Internal = Cid* 
Leaf = (II U Scalar)" 1 



Cid = the domain of chunk identifiers 
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The CM of* chunk h the chunk's unique identifier and maybe ihought Of as the virtual address 
of the chunk in the system. It should be noted thai the d*ma*n-of chunk id's is not related tb the 
domain of structure uid'e which are used to uniedeiy identify tibjkm oh the V*m heap. The 
Header field in a chunk contains administrative wfcrmatkjn about the chunk. fa particular, the 
number of references -tothis chunk in the systems theheight'Of this Chunk in the Vim tree, and 
the high and low indfcies of the elements of thesutemroocetf at thtecb^k are all information 
kept in the header field. If a chunk has no outstanding refeww** to it* *& placed on afreelist 
and its space may be reused when needed. The last cotniiaMtt & the db/a portion of the chunk. 
For an internal chunk, this consists of the Ctfs of its immediate d*K^ dents m the tree. For a 
leaf chunk, this consists of either uM's if the elements of mis structure.are themselves structures 
or scalar values. The size of the data portion is some fixed m. The size of the chunk is m plus,the 
size of the header portion. 

For a complete description of the storage orgamzation of Vim structure, die reader should 
see [17]. In the next section*; we examine the proc^m of making droopy operation atomic 
Structures which are to be copiedirom the Vim .heap onto the backup heapPmust be copied 
atomkally ie, a structure consisting of many chunks snouM be deerrted as being copied only 
when all of the chunks which comprise it have been ttaosftned-onto baekup store and not 
before. In ourdiscttsstotw ;we snail refer to a chunk found in mam store as a VIM ohuhk imd a 
chunk found on backup store as a backup chunk. We shatt Usee*' w>rd*™c/ure to refeftoany 
Vim structure &* array or recent 

5.3 Performing the Copy Operation 

When a base language instruction which needs to augment the backup state executes, it 
invokes the backup system kernel which maintains tttlrrtbrmatkm about the current status of 
structures which_are in the process of being copied. The^ewwl is responsible for updating the 
computation tree and ciommaod Jog on backup atom baled On «fc algorfthms given m Chapter 
Four. When astwctureiito bejcopiedi the kernel c^ the C^fone&m. Thif funetkm takes 
as input the uid of the structure to be copied and produces as its output an acknowledgement, 
copied if the structure was fully copied onto backup store or notcopied if it could not be. The 
latter acknowledgement occurs when a structure (or any of ite substructures) contains eafly 
completion elements. The backup system kernel uses this acknowledgement to determine when 
activation descriptors should be given a new type and when command log entries am be 
removed. 
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A major complexity in implementing the copy, fraction for the Vim system is due to the 
representation of Vim structures as trees of chunks. For the sake of uniformity and simplicity, 
structures found on the backup heap will also have the same representation as their counterparts 
on the Vim heap Le. trees of chunks. At the start of system operation, all chunks on the backup 
store will be on a freelist. Whenever a Vim chunk needs to be copied onto backup store, a 
backup chunk is removed from the backup freelist and the contents of the Vim chunk are copied 
onto it This approach will remove the need forsoprusticated interpretation of the data found on 
the backup store by the recovery system. Activation descriptoi entrief joomprising a computation 
record are treated as records which occupy a single chunk. 

To see how the Copy function can be implemented, let us first consider a simple scenario. 
Suppose that the function is to transfer a structure whose leaves contain n scalar values onto the 
backup heap, where n > m. This structure will be represented on the heap as a tree of chunks. 
Let the leaf chunks of the structure be labeled //, 12,...,$ Clearly, these leaf chunks can be 
copied onto the backup heap without any preprocessing by the copy routine since the leaves 
contain only data. Let 6/,M... M # be chunks on die freelist on the backup store. Then, fee data 
found on // can be copied onto W, that of 12 can be copied to*2 eta After the leaf chunks have 
been thus copied, the routine proceeds with copying the interna* chunks as well. Copying an 
internal chunk is a little more complex because these chunks contain references to omer Vim 
chunk id's, The copy routine translates these references to their appropriate backup store 
equivalents. Since the copy operation is being performed in a hottoBHip fashion though, all 
chunks referenced by an internal chunk would already have been given chunk id's on the backup 
store. If an internal chunk C has references to chunks cf,t2S.i,<m, then mt corresponding 
backup chunk has references to bl,b2, mi .,bm where hi is the backup chunk id corresponding to //. 
The copy routine updates the activation descriptor; to reference this structure only when all 
chunks have been written to backup store, if a fetinre takes place before the ADE can be 
updated, it will appear to the recovery system that no information about this activation was 
recorded by the backup system. Note that ail chunks at a given height in the tree can be copied 
in parallel. 

The copy routine performs a bottom-up breadth-first traversal of the structure. When a 
chunk is copied onto backup store, its backup chunk id is recorded in the header portion of the 
chunk. If the copy routine is invoked lalcr on. to ropy a chunk which has already been copied 
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on the backup store, it can avoid recopying the chunk by first checking to see if that chunk 
already has a backup chunk M. By simply mcofdHig these id'ej the same level of sharing found 
on die Vm heap is achieved on the backup heap as welfc 

In this simple scenario, guaranteeing atomicity is a fairly easy task because the structure is 
not recorded as being copied until the copy funtion transfers all Vim chunks to the backup heap. 
The situation beicomes Rightly rftare cx>mptex when we consider the actions which need to be 
undertaken for copy uig more oon^*cated structures. In partkular v the solution gftert'above is 
not satisfactory for ruuidluig structures whose elwiients are tfiermelvte structures e#*ft array 
of records. Let us Jrstconsidef the(jitiiaiioj*witrioMtiea*ty Suppose that *feaf 

chunk, ik which is to be copied contains references^^to oirw strii«ut« on the heap. It is 
necessary that each of these stieofdinatestnjctu^ 

leaf chunk is allowed *o/ be written onto backup store. Our dtsoription of the copy routine in 
Chapter Four had it recursiveiy catt itself to oapy *«eeau*wruciures. When all strieln^es 
referenced from this chunk have been copied, Ada the kef chunk can itself be trahsfettofonto 
backup store. Because the routine only returns to ^scatter w#en all substructures have Seen 
copied, the copy action is atomic since tile backup state will he updated to acknowledge the 
existence of this structure mfy when me top-level copy mutawcompletes and reuims hi result to 
the kernel. Because we arc ncrt consiciermg earty corapie^ion^ ekraeatSv the Co^ filncdoW is 
guaranteed to return an acknowledgment, copied. No partietty copied structure can ever be 
referenced by any activation descriptor since su<± references sreonfym by the fcerae* after the 
copy operation is oomplete. 

5.3.1 Early Completion 

The presence of early completion elements in the leaf chunks of structures further 
complicates the copy procedure. In Chapter Four, we noted that it is necessary for the backup 
system to be aWe to determine when there are rib further early completion elements still to be 
SET before the oackup state can be updated. In our implementation, we can determine this 
information through the use of reference counts. Every chunk has two reference counts 
associated with it: the refciit is the total number of references to this chunk found in the system, 
and the setcnt is the number of SET instructions still pending which are to set early completion 
elements found in this chunk. The meaning of the refent is the same for anJnternal chunk a& it 
is for a leaf chunk. The setcnt field in the header of an ^rnalcliunk lis the number of early 
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completion elements which need to beset in the tree rooted at this chunk. Thus, therefcnt field 
in the root chunk of a structure holds the total number <©f references to this structure and the 
setcnt field indicates the total number of early completion elements found 'Within the structure 6 . 
When an early completion structure gets SET, the setcnt field in the headers of all chunks on the 
path to this element get decremented 

Let us first consider a smptestructure ie one which does not reference any other structure 
on the heap. To implement the copy operauon in the presence of earty completion structures, 
we do the foUowinfr When a teaf chunk withctef&nt greater than zero is encountered by the 
copy procedure* we do notcopy it onto me backup stor*. Jrnuead, we allocate a backup chunk id 
for this chunk and continue examiniiupthe other chunks in tbfe strtidui*. Tomfofm theSET 
operator which will everuuaily replace the early completion etem«m found in this chunk that it 
should copy the chunk onto backup store, the copy rouune aao febe4s the chunk wfth the tag 
bach. At the time the eady completion element becomes defined, «• sfcr operator wift ribtothe 
fact that this chunk belong* to a structure that is to be espied to backup store. If the setcnt of 
this leaf chunk is zero,, me operate* itwokes the backup ^stew kernel to copy this chunk onto 
the backup chunk allocated for k In addition, if there are no outstanding set instructions that 
are to be executed for this structure, determined fey examining tbe^tcitt of me root chunk for 
this structure,, me kernel also updates the acdration o^scriptor or refetence this structure 
according to the algorithms given in the last chapter. Notice that computation records are only 
updated when all dements in a stnuaure are fully defined and copied onto the backup heap. 
Thus, it will never be the case that activation descriptors are aware of a structure for which rtotull 
elements are known. Atomicity is still guaranteed even with early completion structures because 
a structure becomes a part of a computation record only when no WiM ec-elethenKl are 
referenced from it 

We next examine how the backup system should handle early conation structures 
belonging to structures that are components of a larger structure which is to be copied. Wheti a 
leaf chunk which contains uid's to other structures is encountered by the copy function, the 
setcnt of the root chunk of this subordinate structure must be examined If it is greater than 
zero, it means there are early completion elements which still need to be set, in this structure. As 
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before, the copy routine is recursively invoked to copy the chunks found m this substructure. It 
is necessary that the backup state does not get updated with the parent structure unless all 
subordinate structures of this parent become fully defined. Our implementation follows closely 
with the algorithms given in the last chapter. Every rpot)chuiik of astmeture contains a field in 
its header* Parent, containing the uid of all structura which reference this structure Aatare to 
be copied. This field is mangaged by the copy function as ittraverses the heap. Associated with 
each uid in the list, we also store the chunk id of the fteaf chunk in the parent structure from 
which the reference emanates. When all early completion dements are set in a structure that is 
to be copied onto backup store, ail structures referenced iaJts Parent list are examined The 
backup state is updated with those structures referenced in this list which have become fully 
copied. Determining whether a structure has been fully copied involves keeping track of the 
number of chunks that still need to be transferred and the status of its substructures. Counters 
are used to record this information. Every root chunk, in addition to the Parent list, also 
contains a ToWCopted counter indicating the number of chunks still to be copied onto backup 
store. When this value becomes zero, the backup state can be updated. Every leaf chunk also 
contains a counter indicating the number of substructures which have not yet been fully copied. 
When this counter reaches aero, the leaf chunk can be copied onto backup store and the 
ToBeCopied counter can be decremented Much of me detail involved in implementing the 
incrementing and decrementing of these counters is not very interesting and is omitted here. 
The important point to note with respect to earJy completion is that the backup system kernel is 
responsible, not only for updating the backup state whh the structure containing the early 
completion element being set, but also for updating the backup state with all of its parent 
structures that need to be copied as well 

5.4 Storage Management 

One other implementation detail that deserves brief mention is the manner in which 
storage management is handled on stable store. The c>rganiraUOT whims ehoseii Ibr stable store 
lends itself to a very simple and efficient storage management strategy. Recall that stable storage 
is used to hold transitional data represented in the fomi of compuuukw record! Each 
computation record embodies the progress of some active computation in the system. When the 
result of a computation is recorded on stable store, the associated. computation recorded can be 
deleted. Deleted computation records arc added onto a favlist of backup chunks. We expect 
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that the set of quiescient data found on stable store wHrheperfc^fcally written onto tape. After 
the transfer, the space occupied by these data items m stable store can be reclaimed. It is not 
necessary, however, to wait for the transfer of information to tape beforespaec -can be reused on 
stable store. Once the result of a computation is bcwrtd man environment, the storage occupied 
by the activation descriptor entries of the corresponding computation i*cord*catf be reused. We 
use a variant of a mark and sweep garbage collection policy to reclaim structures referenced from 
activation descriptor entriesiir a computation record that mtt%&f*tmne&: Once the result of a 
computation is known and is copied onto the backup sfo^ «$ ehunks in that structure are 
marked. We can then reclaim all chunks belonging to structures referenced from ABB* in that 
computation that are not markett 

The storage management policy is very simple fqr our system because we can determine 
precisely when data is no longer accessible by observing the djnoamicfof the command log — 
removal of an item in the log implies that the associated computation record lean be reclaimed. 
Waiting for a computation to complete before reclaiming the storage it occupies on stable storage 
obviates the need for introducing a complicated garbage collection impolicy on stable store. . 

It may be the case that the value of a computation is recorded on the backup environment 
even before ail the values of t^/<Z)Fs in the subtrees of me computation are. ISmee me result 
of the computation has bees recorded in the en viromnent the associated computation record can 
be placed on the backup store freefoL Removing the oomputttion record, however, necessitates 
taking some action to inform the activates wtose results wouten^ 
ADFs that the computation record is no longer part of the backup heap: TO do this, we 
maintain a uid entry table which contains referencing iftfbrmatiofl for ail activations in the 
system. Entries in this table include the uid of the activation, the address of the activation in 
memory and the address of its ADE on backup store and a referent! to 5 a booleSrV flag which 
indicates whether the compulation record has been reclaimed or not When the computation 
record is removed from the backup state, its associated flag is set to true and the result value is 
not copied. Only the root activation of a computation cart change mis flag. Clearly, this 
operation is outside of the purely functional programming paradigm thus far used: such resource 
managers are to be written usmg the guemtton construct/JOf wWch aHows this sort of* non- 
applicative behaviour to expressed man appifeativefsetting. 
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Chapter Six 
Conclusion 

This thesis has proposed a design for the ¥lM computer system which guarantees the 
security of all online information against loss or corruption ass result of hardware failure. We 
developed backup procedures that a» imeadeoVi to, execute ^concurrency with normal 
computation. These procedures record the progress of ,4*1 computation in a compact fonn on a 
backup storage medium. When a compulation completes, its result a bound tosome name in 
the Vim users' environment structure. Once this binding is added to the backup environment 
image, the computation record associated with;; Oris computation is removed from the backup 
state. We make no assumption as to the integrity of.anytdaitowhichaimws a failure except for 
data found on the backup medium. The backup medium consists of two main devices: tape 
storage used to hold ah* data bound to identifiers in die user environment, and stable storage 
which contains information in the form of computation records about all active computations in 
the system. When the recovery procedure is invokecl after a failure* fc first restores the Vim 
environment structure using the backup environment image. It men begins reexecution of those 
commands which were executing at the time the failure occured. the time to reexecute these 
commands is greatly reduced because of the information kept on the corresponding computation 
records. Once all commands have been reexecuted; the recovery process is complete and the 
system can accept further commands from its users. 

6.1 Contributions of the Thesis 

The contributions of this thesis have been two-fold. First, we developed backup and 
recovery algorithms for the ViM system. These algorithms are very different from those found in 
more conventional systems. The unique features of Vim that required a novel approach to 
providing data security lie in its applicative programming model and in the use of a uniform 
representation forljoth data and programs. This homogeneity eliminated the need to maintain 
distinction between files and data, thus allowing the backup system to be more closely integrated 
with the ViM interpreter than would otherwise be possible. The use of an applicative base 
language made it possible to have a simple organization of backup store because data in such a 
model never changes its value. We exploited the expressive power of the base language 
instructions by distributing the logic of our algorithms among the salient base language 
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instructions. The apply operator, responsible for creating a new function activation, was 
augmented to append to the appropriate computation record a new activation descriptor 
corresponding to the activation to be instantiated, Similar enhancements were made to the 
RETURN operator and various structure operators as well. By embedding these algorithms within 
the interpreter itself, it was possible to achieve a measure of dataseeurity far greater than what is 
possible in conventional systems. In addition, such a design makes the operation of the backup 
facility completely transparent to users of the system. Similarly* once the recovery procedures 
restore the system state after a failure, subsequent imputations will not be able to determine 
that a failure occurred by examining the restored state. The fact that the base language is 
applicative also freed us from having to introduce complicated backup storage policies. Data 
once written onto to backup store is never updated? it is either garbarge collected or bound to a 
symbolic name in the backup environment 

In order to rigorously specify these algorithms, we developed a formal operational model 
of system behaviour for Vim. This model views VlM as a state transition system with the Vim 
interpreter being a state transition function. We presented the definitions of some of the more 
interesting base language instructions in this model using a variant of the functional language 
VimVal. This basic model was later enhanced to incorporate the backup system as welt The 
backup state was treated as a separate component of Vim. The interpreter was now treated as a 
state transition system on a two-tuple consisting of a Vim state and a backup state. Beyond being 
an important tool for expressing our algorithms, this model also allowed us to give a formal 
proof of correctness of our algorithms (presented in the Appendix). 

6.2 Future Research 

One important area of investigation that was not addressed in this diesis is the issue of 
correctly preserving the state of non-determinate computations. A non-determinate computation 
is one which may exhibit different behaviours for the same inputs. This type of computation 
contrasts with determinate computations for which repeatable behaviour is guaranteed. A major 
assumption made in this thesis was that all computation was determinate. This allowed us to 
design a recovery system which can reexecute computations whose results are not recorded on 
the backup store by preserving the arguments to the computation. Such a design is possible 
because any computation in this model is guaranteed to produce the same result when presented 
with the same inputs. The basic non-determinate operator in the Vim base language is the 
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MERGE instruction. The merge operator is enabled whenever an input arrives on either of its two 
input arcs. Thus, the behaviour of this operator is characterized by the arrival order of its inputs. 
Such behaviour is inherently non-determinate. 

An important area of future research is augmenting the design proposed in this thesis to 
handle non-determinate computation. The changes made must take into consideration the fact 
that non-determinate computations cannot simply be reexecuted by the recovery system since 
there is a multiplicity of output behaviours possible. Reexecuting such a computation with the 
same inputs is, therefore, not guaranteed to produce the identical outputs. 

Most transaction systems such as airline reservations, banking, etc. are based on non- 
determinate computation. Such systems also typically have very high data security requirements. 
Enhancing the Vim backup and recovery system to support non-determinate computation would 
be an important step in understanding how highly secure transaction systems can be written in 
an applicative computer system. Issues of atomicity and indivisibility of transactions could then 
be addressed in this context 

It would also be a challenging task to map the abstract specification of the algorithms given 
in Chapter 4 into an efficient implementation on the Vim system. Realization of these 
algorithms will require optimizations not addressed in the thesis. Such optimizations include 
minimizing the amount of copying done to stable storage and efficiently reclaiming storage on 
the backup medium. These problems which were hidden in our abstract model were briefly 
addressed in Chapter 5. A truly viable implementation will need to confront these issues in 
much greater detail. 
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Appendix A 
Proof of Correctness 

In the previous chapters, we have developed a formal model of the Vim system to describe 
our backup and recovery algorithms. Beyond being a convem^fvcllJcleirt'wnleh to precisely 
state our algorithms, the formal model can also be^sed^ptovln^^cor^^ of bur design. 
Intuitively, demonstrating the correctness of the tiirtitfMifeftttotelry ptocedufesihvolves 
showing that the recovery procedures do not resti^ft inedfrc& stdteliasedotf the information 
kept on backup store by me backup procedures. - TW «6mj*utiiMohs wWdt execute after the 
recovery procedures complete should not be able ton^i#lhel&ftHaM'a falmre had occurred 
before. Thus, the state of the system observed by any computation instantiated after recovery 
from failure must be equivalent to the observable state which Ifcfetea* plrlbr tothe failure. In 
Vim, computations observe the effects of ©the/ co|»pu^tions u^ugji d^ V^! environment 
structure. It is, therefore, necessary that the environmem.image wstortd by the#ecovery system 
be equivalent to the environment whkftttdsfed before t$e;fi$re\ We present a formal 
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definition of environment equivalence later in this appendix. To establish that environment 
equivalence is preserved, we examine the s|ate ^.^aQj^o^jjr^uced^byrtiw system when 
interpreting the command tog daring recovery and compare lt*itlrthe state transitions that 
result when interpreting the same log when no backup state information is used. To show that 
environment equivalence is preserved across the two transition sequences, we use^he fact that no 
instruction is executed during the recovery process that would n^abfii have been executed by 
the VIM interpreter evaluating the same command wheja. n^ u^raia^^n the backup state is 
utilized. Our proof is as follows: We first examine the state created by executing a particular 
instruction in the transition sequence of the recovery process. We then demonstrate that this 
environment is equivalent to the environment coinp^hehf of Ihd sISte created by executing the 
equivalent instruction under normal interpretation, tishigi sfrnp^r inmictlbn argument, we then 
show that the environment component of the finer rco^vcTysfcttefc equivalent to the 
environment component that woufd have resulted If nb Failure had tiutetr' place. 

In the following sections, we formally define our nptipn^ of transition sequence and 
equivalence. We then prove our main theorem: The system state after the recovery process 
completes is equivalent to the state that would have resulted had ho failure taken place. This 
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implies that it is not possible for any computation to observe the effects of a failure once the 
recovery procedures complete. 

A.l Definitions and Terminology 

State Transition Sequences 

Because we are considering Vim to be a4etermin#te system, weoiuvmodeJ the execution of 
a program as a sequences of states. The Execute function which whiffiigiMmacujreM state and 
enabled instajction tetiiins the state cjeatedrby exeeatiiigTih*^ 
of the base language instaictioiisajMtffe^ 

using the normal or xecoytMy interpreter, In ovyp proof, we slwril be cooceroed with examining 
state transition sequences constricted by the respective interpreters on the command stream 
preserved on the backup tog. 

Definition 1: The System State is a two-tuple, XVimStaie, BackupState> where 
VimSiate and BackupState vttn defined in Chapter FoW; Tlw recovery command 
stream for a system state S is* se^nee of oofn^jpo^ <c / ^.„,c A > where 
BackupState = <Log,BHeap,B£nv> and Log(i)xornmand = c f 

Definition 2: A state transition relation, 1-, is a relation on states. Let S - 
<ActJI t ElS,Env> and T = <Act',H'£tS!MmX Then, S fcr, Tiffi 1 -(«£ € £/S ** 
Execute{S,(uM = T. 

Definition 5: K state transition sequence, <S t S?...,Sj>, is a sequence of states 
such that S / K S^ for 1 £ I £ M. 

For notatibnat convenience, we shalT sometimes denote a state transition 
sequence <S f $ r ..&>m Sj&S* 

The recovery process is divided into two phases, la the first .phase, the contents of the 
backup environment are restored onto the Vim enMUW»n|§f^.sKuc|i ! ire. Oftce this is done, the 
command log is reexecuted; during this ^execution phase,, backup iiUormation is used to avoid 
unnecessary recomputation. The state of the system after jUte fir$ phase completes is called the 
initial recovery state. The heap in the initial recovery state contains all structures referenced from 
environment entries in the backup environment image. 

Definition 4: Let S k be a Vim state such that Failure (S k ) = true, Let B K the 
backup state corresponding to S k . be <Log, BHeapBEnvX 
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Then, the initial recovery state for S k is the state: <{}, H ff {}, Eriv^ where 

HJ^u) = if 3 namest. BEnv(name) = wand BHeap\u) = v then v. 
= if 3 R,U r ms:t. {Hfiij) a R) A 

<(#fe«) * »&A (BHeap(u)= v)then iti ■. .', , 
= wflde/otherwise. 

Env Q = BEnv , 

Unlike iqstructi^, exerted d^g^noxra^pj^tk^^^^ during the 

recovery process use information found in the bactopstate. th^Al'ttiY^wtfUction, for example, 
avoids instantiation of hew activations if the value for an ac|iyj||ioii w,bich, previously executed 
has been recorded on backup store. The comman#iogcfteft^n^lirt)fl^up^st^e is a sequence 
of commands whose final results have not yet J^Jflj^J^^^a^^MC-^*^©^ ^fft.MP environment 
structure. When a failure occurs, the recovery system reexecutes the commands found on the 
log, using backup state IHfoimatiori it has accumulated about me ca^f^tipn^ To demonstrate 
that the recovery process interprets this information correctly, we examine how the Vim 
Interpreter would execute these same commaa/ls : if aa backup ^einfonnatioa is^used. The 
transition sequences produced by the respective interpreter^ v is known as a transition sequence 
pair. If the recovery system interprets the informatioo ib«d on ttee Wokup state correctly, it 
follows that the final states fii- 1J tfie~ t Mirci' CriuiAlskte4i^dSllii^^iniM^;' t itti^~'^ni^|«leht environment 
components. 

Definition 5: Let R be the recovery command stream fqt stated and let STS = 

<s y S 2 *V ** ** State transition seqoeiice#pRd«wsb*«i8ft **nevaiuati©n of the 

call: ShelKR, S /t <{},{},BEnv>\ Then, £ is the initial recovery state and S f is the 
ideal recovery state for 4?. £lSj.&tf>, the mlplF>mfr*1n& k*Sffletf ^tfte standard 
trmsttion sequence fix system *deS. ■ ■■■>■"■* '*» -■?>;-. 

Let R be the recovery command stream for system state S and let RTS = 

<R V R 2 R ? ** me MM* transition sequence produced tftfring the evaluation of the 

call: Recovery(<Log,BHeap,BEnv>). T4i«,« AJ%it^$aFfS6^^ 

the fina^nxowff state for *.L|/fc is p, *#?eto#y tot? kfs Stalled Hit recovery 

transition sequence for system state 5. '- 1 

Definition 6: A sequence pair for a system state. £'isYfcfcHupfe: <R T$,$TS> 
where RTS = </? 4 J? a . t ..^> and STS = ;> <&j& r .*S^y \£ k wh^re/Lasd^are the 
final recovery stale and" ideal recovery state for the command stream, resp. R } = S v 
the initial rcrovei^^tite for system stirte 5:^^^^^^" * ' ! f : r 
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State Equivalence 

In the following definition, we shall use the symbol, s, to denote the equivalence relation 
between corresponding state components in differerjUstatesl Thus, if Env ( and Env. are two 
environments in states S and S', then EntjS EwM&esb two- environments are equivalent 

Definition 7: Instruction Equivalence: Let FA and F^'be two activities. Then, 
FA(m) a FA '(«) for m,n € N if: 

•FA(m).opcode - FA\n).oipcote. 

•V opnttm*. fOpmAmi,opftom2,oipnttrrl3}, FA(m):oprium - FA\ri).opnum if 

FA(m)j^numan^FA{nhopmm ■€ Scafc^ 
•V o/wwm € {opnumi ,opnum2,opnum3}, FA(m).opnum .m FAXn).opnumif 

FA(n0.opAum^FAifi).opnunttl}. 
•FA(m)loper& = FA'{n%opcr&. 
•FA(m).s\gcnt =? F^'(/i).si^pm. 
•V (dc,k,opnum)e FJf(i).dest, 3 (de,l,opnum) in "FAXjijteSt it F^(/fe) = F/i'(i). 

Activity Equivalence: Two Activities /V4 and F^'are equivalent if for every in £ 
N,FA(nffsFAXm). 

Object Equivalence: Let /^ and H, be two heaps defined such that Hfu) = v 
and Hpi') = v'for «,u'€ U. Then, v a r"lf 

•v = u/u/e/ v"= ande/ 

•v € Scalar, then v'€ Scalar and v= v'. 

•v € Record, then v'£ Record and KO s v'(/), for every /€N. 

•v € ECO, then v'€ ECO andV (u k ,i) € V, 3 (u.m) € v'it (u.,/) s («,/«). 

• v € SUSP, v*€ SUSP s.t. if v = (w ,m) and v* = (« n ,/t), Acliu^m) = 

^c/(«J(«) OR ///v) € Record and v'€ SUSP and 3 state S k where S*HS ft and 

H k (v)& Hjif). 
•v € U, then v'€ Uand ff/v) a Hpy 

Environment Equivalence: Let Env j and Env, be two environments in states S ( 
and Sj Then, £'/iv / and £/iv. are equivalent if for every *tj a£ Emfn t ) = v 3 a^ it 
£/iv(/»p= v where if 

•v = undef, v'= undef. 
•v € Scalar, then v'C Scalar and v av'. 

•v € U, then v'€ U and Hfvjm Hh"). H { and //\ are the heap components in 
states .Sj. and S, resp. 

Containment and Completeness 

In order to show that environment equivalence is preserved between states in the recovery 
transition sequence and the standard transition sequence, we will need to examine the effect of 
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executing equivalent instructions in the two states. Thus, it is necessary that for every enabled 
instruction in the recovery state, there be an equivalent wstructiqn in the corresponding Vim 
state. This property is called containment. 

Definition 8: Let <RTS,STS> be a sequence pair for a system state, S, where R ( 
is an element of RTS and S, is an element, of STS. ^ t » epiuqf/i*t-mS t i&V{wn) € 
EIS l 3 (u',n) € EISjiL FA\m) a FA'(n) where^c//tt) =j£$pA4ctfui= FA', 

Definition 9: If <RTS>STS> is a sequence pair ft** System state, 5, and A,is an 
element of RTS and SJ* an element of 3f$'dMirtR]fff «mpfe/e wit S\ if £>7y, a £hv. 
and /? / is contained in 2?, 

Computation Sets 

In Chapter 4, we introduced the computation tree as an ^bistraction to describe the 
instantiation of function activations in* computation. The following three (fefimtioiis formalize 
thisidea. 

Definition Uk Let an activation a be instantiated in state 5. Then, the 
computation set Cof a is defined as follows: 

•a€C 

•Any activation instantiated from an activation in the computation set C is also 
inC 

That is, the computation set of an activation (im»> represents the transitive closure over all 
activations in the computation tree rooted at (M- 

Definition 11: A computation sequence CS for an activation instantiated in state 
Sj with eomputatioii set C is a state tn^m?MQMm&SzM£^:,S 1 ? where k is 
the smallest integer such that none of ElS k ^ J ^E4S^90mmm^mpmkms from 
activities found in C. ." 

Definition 12: A value, v € Scalar U ST, js accessible m a state <Act,H.EfS,En v> 
if either: 

•There is some activity A s.L A(n) = I and Jppnum = v for op/mm € 
{opl.op2.0p3},ahd^c/(i/) = ^. 

•There is some activity A s.1 A(n) = I mui I.opnum = w for opnum € 
{op\JBp2jop3lAc4u')s A and H(u)^v. 

•There is some v'€ Record s.L v'(m> = vor v'(m) = u where A/(m) = vand /is 
accessible. 
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A.2 Proof of Correctness 

Lemma 1: Let R. = <Act r H p EIS it Env^ be an element of RTS and S, = 
<Act f Hj,EIS.Env> be an element of STS. Suppose^**- R UJ on an enabled instruction, («,/), 
where Actfjt) = FA and /v<(/).opcode * apply. Then, If R ( is complete wit S, 3 a state 5^ i/i 
SyH-S^ and /? / ^ / is complete wrt^. J 

Proof: To prove the lemma we need to show that environment equivalence and 
containment holds between R, +J and some state $ k for any instruction whose opcode is not 
apply. We prove the lemma by examining the different classes of instructions defined in our 
model and show thai the Jeiama holds over each of these classes. Jfrjf^complete wrt S. then, 
by the property of containment, there «0sts an eiaWe4«striK^oJi,^aV>€ ElS.m («,/). 

1. Scalar Operations: The effect of executing a scalar operation does not alter the 
environment and so, both environments in R. +] and S, +/ remain equivalent By 
definition of instruction equivalence, we know that every destination (dcj,ojmum)te 
FA(i).dest s some destination, ((kj.cynjwtti m. t FA'(^f^, Suice the same scalar 
values are sent to equivalent ihstmctions, tfiese destinations remain equivalent 
Thus, equivalent instructions become enabled ffiW^WMPm^ Hence, ft f+1 is 
contained in S,+ 7 and, so, the lemma holds for scalar operations. 

2. Structure Operations: A structure operation performed in state R, might cause the 
new state fl (> ; to have -a different heap* Note mat the envinoi^nefrt component does 
not change when any structure operation executes. 'fo^ee'AaVthe letfrninstBI Wolds 
for these types of instructions, we examine the various structure operations in our 
system. 

a. CREATE: A create instruction executed in R ( will produce a new structure on 
the heap in R (+ . with uid i/ ; and size n, with ail elements of the structure 
having value undef, where n is the operand to the instruction. The equivalent 
instruction in Emfivf&vtomwmi^^ 

the heap in S, + , with uid u 2 and size p, T f^^u* d^flmti^okheap equivalence, 
H. +1 and fr J+I are still equivalent Showing that R i+t is still contained in 
S J+ j follows the same argument as given iaj|^a*e, 

b. SELECT: A select operation ca» be performed on a structure element mat is 
either^sciriiu-cbafttcmtef value. v aft early compJetioftttructure or a suspension. 
We examine each of these in turn: 

i. value: There are two cases to be considered if the item selected from H { is 
a scalar or a structure. *i the rlr^ea&MefteW selected from H. would 
also be a scalar or structure value. By the property of containment, the 



operands to the select instruction must be equivalent. Since the heaps 
and environment components do not c|a<jge m /^is^se. M (H m H ,. 
R i+I is still contained in S. + l because all ofetiriatiorts of the SELECT in 
/?,. are equivalent to*le&inatiO«SinJ& and. (blowing the argument given 
in (1). these instructions would) remain equivalent TTuii R^/ is 
complete wrt S.+ / in this case. 



§A.2 PROOF OF CORRECTNESS 



In the second case, the item selected in ff ( i9a record represerrting a 
stream, but theconsespondingiienl'hl W. is asu&pensiOhV This case arises 
when me afply operator is to instantiate aTunetiOrt represented by a 
scream Ad*. The stream 4male>^ ? «ri^^re ' ^restored by the 
recovery procedure, irtstaeatf the awpcnsion triggera the^tantiation 
of a new activation. A, to |toe^ the new stream element Let 

^j+f^lf r- -&? bea «*** ***^Hibim&<i^jft!hmvfa all 
enabled instructions i»^/Sy^/«SSe|rt#lo^ belonging to A (Or any of A's 
dependents) in the compufau(» w« WOtetf at '^. TOeft.W'wffi contain 
the new stream element and, by the JJrooem of object ejqutvalence* the 



two streams m'fy? y arid M n wduttm #/*&!& ^ is obviously still 
contained in S n . Thus, R i+I is still complete wit S^ 

ii. early completion: If toe item sctected from ^ is an early completion 
structure, then the select instruction, (u,ilis added onto the queue. Since 
operand values 1 are equtvatent (becau^ faWMthMtiij, the select 
operatk* performed in Sj woolrtateorariett an*ariy completions structure 
from H. The select instructionstabpth sUtts wouW get added to, the 
e-queife. Sttce ho new irisulict^'becSnle^Dled; %^ ', is contained 

mS j+r _ _ ..^1 U ' IT;- ;; ' ...;'.'■;' ■ 

Hi. suspension: If the item selected from H t is ^suspension, the instruction 
'reifeTeticed hi the suspension V s^anaje^ . ^^ ^r ^ p^pperty pf 
containment, the item selected in S. must abo be a suspension and the 
instruction referenced in lhip j i ^enijqn : would ate be signalled The 
equiva^lrwiruftioft would m^fitlvM^^^^^mdr^imd,^^ 
instruction equivalence is stitt pre*»raeoV Tie executioo of the 
susp^ns^ ; o*«H^ itJfh$ an ,«ei*r completion queue in 

bc^*a^coj|iain^ 

operators^ 4^1$ ,&» a« ce^wataw, ^fady-oompteuon queues are 
alsoequi^u^T^ 

c. SET: By property tf oailiJi^ equivalent to 

the One SET using 7/, ^addition, every y^^SXj^^^^'^fi' 
value in H i is equivalent to an element in the ec-queue set w§J%e : value in H. 
Thus, afr^r *cse^u^S becom 
presetted ahd!lm$ /T, * %!fflb%^f&^™ 7 ~,. 

3. Terminate Operation: Execution of the terminate instruction causes control to 
return to the arteH. with the ^ environment 
Since #f**bm^4#>6^rvj&lM Wwmtim& mtaetibtt J rnW be 
equivalent in both R ( and S f Env , is, therefore, equWs*HH&fcdV^;; Fbllowtttg 

^ the same argument g^ 

4. Return Qemlk>a: T*e return mstructkai lakes fwx>*fg*mcm me return value 
and the target list fey the property of containment the return value and target list in 
the equivalent instructions in R. and S, must be equivalent. R u , is contained in 
S y+ /sfcKfeihe u«rgcti»tfdresscs of thd r^ttrrri 1 ttiSbrdfer ihSi ! c4jii^Mnt iflfcf the rxy^lt 

values are equivalent in the two states. 
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5.Tailapply Operation: The tailapply instruction takes in three arguments, the 
function closure, argument list and return link. By the property of containment, all 
three operands must be equivalent in the executing instructions in corresponding 
states. The result pf executing the instruction Js toadd anew activation and to signal 
instructions in its, own activation. Because; the, closures and argument lists are 
equivalent, the instructions enabled in the new activation must also be equivalent in 
R i+I and S J+I . Following the argument given in (1), it is easily seen that R. is 
contained in S A similar argument can be applied in the analysis oTthe 
streamtail operator and is omitted here. 

Since we have shown that the lemma holds for all instruction classes (excluding apply). 
the lemma is proved. □ 



The effect of executing a function is visible only in the result value returned by that 
function. Thus, given a compulation sequence for some activation, the values that are still 
accessible after all activations In the computation sequence have completed are precisely those 
returned by that activation to its caller. 

Lemma 2: Let <S,S. 5^ be a computation sequence for an activation A. Then the 

values accessible in S k+J , but not in S'are those values acc^blejrpmtne instructions found in 

ffiedesttharinri bet nf tfa» ortivarinM J ■■••.-. 



the destination list of the activation, A. 

Proof: (bycom/adtetidn). Suppose there is some vahie, n that is accessible in 5. .but not 
in S and is also not accessible from the wturn link to trtf activation. TthV value must nave been 
created by some instruction thai « an element of some activation to me computation set of A 
Let this activation be o. In order for this structure «5 b*att*srfrite alter me termination of this 
acuvation, it must be returned as a result of that activam*inc*%iy references to it from 
instructions within or are lost oneethe RmEAS€msmictid« %xmti& m itscaller be B. In order 
for the value to be accessible in S. ,, h must, by the same reasoning as above, be returned as a 
result of this acuvation as well. Continuing this argument we see tiuovthe, value can only be 
accessible in S^ if it Is returned as the result of A. But this contacts our original hypothesis 
and so the lemma is proved: D ~ ™ 



c ^ X h o e0 ^ U } M t* 7 *^* t* > sequence pair for a syst^ state $ and let J?, € RTS and 
A y € i>TS. Then, if^h- R and 7? Is complete wrt 5, wen 3 a state S. U S/h-kand R, , 
is complete wrt S k ./ * «■ . /+/ 



fl ii M » | . ii i fkm 



Proof: Our proof is by induction. The induction step shows that the theorem hohte over all 
instruction classes for our system. 

Basis: Let RTS = <R r Rf and STS = <SS>. Since /f. = S r and by definition of the 
initiaj recovery state, £/5, = qp, R, = R .and S/^ S Thus,/*, * feartd the theorem is 
satisfied, J J . A I 

Hypothesis: Suppose that the theorem holds for aU sequence pairs <RTS,STS> where RTS 
is of upto length i.i>2. 
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Step: We show that the theorem holds for a sequence pair <RTS,STS> where RTS is of 
length /+L 

Non-Apply Instruction: Refer to Lemma 1. 

Apply Instruction: There are four types of actwastion descriptors that are found on a 
computation record. These descriptors can ettiwr be of type: apply, value, tattappty or 
streamtatt, When an apply insmiction executes in A. it exarnmath* activation descriptor for 
the activation to be initiated. 

1. apply or nonexistent Ade, If no Ade exists, then state ^ /+/ is complete wrt state 
S^j since the apply instruction executing in '..state ^u^no backup ^formation. 
Since R. is complete wrt Sj, and equivalent jnstriwsi^ these two states, 

then by analysis similar to the one given for the TAILABPLY, operator in Lemma 1, 
R t +i and Sj+j remain equivalent This sjune analysis holds when the apply 
operator is to instantiate an activation which has an activation descriptor entry of 
type apply. Since no result is found in the Ade, anew activation is created by the 

APPLY. 

uolue: If a ualue Ade exists for this activation, then R^ . differs from R t in that the 
value v is accessible from an activity in R ; but is nm accessible from this same 
activity in R f The equivalent instmctio^ ex«iUtin| jfii \"S, would cause the 
instantiation of an activation, (u p A\ because no backup mformation is used. By 
Lemma 2, there is a state transition sequence <SJSL^^^ such that the values 
accessible in S J+k but not accessible in &*&&em Twlues accessible from 
instructions found in the destination list passed to the activity, A. These values 
represent the result of the activation and thus must be equivalent*© the values found 
in the Ade of the activation used in R { (since we have no non-deterministic 
computation). Since the apply mstmctioh executing % sends Hit lvalue of the 
activation found oh die Ade to all dettmatrohWantfbf ^th^ptp^rot containment, 
the two apply instructions executing fa R . aftd^iia% e^rValent destination 
instructions, state * /w must be contained iristfcr^.'' Since °tte envlrirtment 



2 



image does not cna*** in either ^state, R Ui is complete wit S,^ and, so, the 
theorem holds foYthfc^ceje. ; j+* 

3. taUapptu: If the Ade for an activation has type tailajjply. then %, differs from 
R ( in that R^ } will contain a new activity. A. whose argument record is retrieved 
from the acttvation descriptor. When the corresponding apply instruction executes 
in S f it will cause a new activity, A ', to be created whose argument record is not 
necessarily the same as in A. Consider the state transition sequence, 

<S f S j+/ • s y^. m > whcre s i+ m ./ •- S j+ m Qn a tailapply instruction which creates a 

new activity, 5. such that B s A. There must be such a transition sequence since all 
computation in the system is determinate. B is a descendent in the computation tree 
rooted at/*: Fix the transition sequence from S, to S + m so that EIS . contains no 
instruction from any activation on the path from A'lo B(or descendants of any such 
activations) in the computation tree rooted at A'. This is clearly possible because of 
the non-dctcrminacy of the Choice function. The only new values accessible in S. 
not accessible in S. are those referenced from B. Since BsA.R f h still contamc3 



■tip .jf,«V^*^l 1 .t «* *-*^ .-' ■*•""■" ^* , *»'Jp*«A«ia.-*' 



120 PROOF OF CORRECTNESS §A.2 

in 5 /+m . Since the environment image is not updated ineither transition sequence, 
environment equivalence is still preserved. Thus, R i+] remains complete wit S, +nf 

4. Streamtaib The case when the Ade Is of tyfle STR£Al^flh$ «ery similar to the 
tailapply Ade given above. In this instance, R ( . differs from R. in two ways. 
First, the stream image on stable store is restJM^ omo^, 7 Secondly, a skeleton 
activation is created to instantiate production of the rem»rH*er of the stream. Instate 
S. the apply opei^torwill<auseanewacuvatk» ¥ >4/tobec^atedtar>roduceme fim 
element of the stream. To satisfy the property of containment, we consider me 
transition sequence, <S,S. r ...,S, + > wbere £/& cpnfains no instruction from 
any from A (or any of its descendetm) in this cd^uta|ft|h $j$ iobted* at A. At this 
point, the first stream element is completely detSAetfaTld there is a suspension in A 
which is not yet enabWd to piodttce the (^thM^ B| ^ a^r^ition of object 
equivalence, me corresponding streams in S.+ m and R]^. are equivalent Thus, all 
destination instructions of the apply in the correspond^ states which become 
enabled also remain ^equivalent. Hence containment y^rpm Wwe^ ^ . ^ and 
R i+J . Since the envhtmlmeht image does not change, %?j re^aini J compl|te wrt 

S J+rri 

Since we have shown the theorem for each of the classes of Affe's which may be found on a 
computation record, me theorem is proved, a 

Corollary: If <RTS£TS> isasequence pair, diett ^.is corhplete wit S^. where U,and S f 
are the final states for RTS and STS respectively. 

Proof: (by contradiction) 

Suppose that R,was not complete wrt to S, Since £7jy ^ ;$>* w^jknow that/Lis contained 
in Sy, It must, therefore, be the case, that en vircmn^teo^i valence, is aot preserved between the 
two states. By Theorem X we inow uwt ^mu^^compi^ wrt sewne qtate S, where S, t-Sj 
As a consequence of the hypothesis, there must, b^rfo^, environment aliefing instruction 
executed in the transkionseo,tf|M^^S, t ^ This 

is a contradiction since by definition 5, the same command stBeam^;ii«ed « producing jboth the 
ideal recovery state and final recovery state. Thus, our hypothesis that RM not complete wrt S, 
must^fa1seafldl^ec»»cMlaryterirtwe^ / d ' : ■ :1i - i? '*» ■ •-• ^ '**■ "* "' -*' : "- ' ' r -- ' 



44. 
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